Conditional param's

I just wanted to confirm how one deals with a parameter which has a different distribution depending on the value of another parameter, i.e. in the case below (using CatBoost), I have to limit max_depth between 1 and 8 when using Depthwise or Symmetric grow_policy but can use 1 to 16 for Lossguide

Can I use the same param name for both branches? In the case below I named it differently as I was getting a warning. The problem then though is at the end of training trial.params contains the name arguments of suggest_* not the hparams keys.

    if hparams['grow_policy']=='Lossguide':
        # Max leaves is only used for Lossguide grow_policy
        hparams['max_leaves'] = int(trial.suggest_loguniform("max_leaves", 2, 64))
        hparams['max_depth'] = trial.suggest_int("max_depth_16", 1, 16) # or trial.suggest_int("max_depth", 1, 16) ?
    else:
        hparams['max_depth'] = trial.suggest_int("max_depth_8", 1, 8) # or trial.suggest_int("max_depth", 1, 8) ?

Also, in the case below grow_policy must be ‘SymmetricTree’ when ‘boosting_type’ == ‘Plain’, so I didn’t sample grow_policy in the else branch. Problem then is it’s not part of the trial.params and I have to apply the same logic to infer the correct hparams for the best trial. Should I use suggest_categorical with a single category as per the comment?

    hparams['boosting_type'] = trial.suggest_categorical('boosting_type', ['Plain', 'Ordered'])
    if hparams['boosting_type'] == 'Plain':
        hparams['grow_policy'] = trial.suggest_categorical('grow_policy', ['SymmetricTree', 'Depthwise', 'Lossguide'])
    else:
        # On GPU grow policy Depthwise can't be used with ordered boosting
        # On GPU grow policy Lossguide can't be used with ordered boosting
        hparams['grow_policy'] = 'SymmetricTree' # or trial.suggest_categorical('grow_policy', ['SymmetricTree' ])
        hparams['has_time'] = bool(trial.suggest_int("has_time", 0, 1))

Or am I just doing it wrong, should I be attaching hparams to each trial somehow, since trialparams isn’t what I’m actually training the model with? I’m used to hyperopt which had a way of recreating the hparams from the space and params.

[HideakiImamura]

[FYI] We are planning that we make FrozenTrial can be used as FrozenTrial.suggest_* samely as Trial.suggest_*. Ongoing at Issue https://github.com/optuna/optuna/issues/1251.

[david-waterworth]

Thanks I was just about to ask that, I’ll close now and follow that issue!