'TPESampler' object has no attribute '_group'

The bug appears as soon as I run optimize method on a loaded study.

Strangely, the study worked today (2021-06-07) a few hours earlier but when I wanted to continue it further, the bug started to appear. Interestingly, the study that works fine on v.2.5.0 which I have on my local machine and confirmed the study working on Colab with v.2.5.0! I think the study was saved by the earlier version than the current one, 2.8.0, and there is something wrong with 2.8.0.

Expected behavior

The study is being optimized without the error.

Environment

Colab, Optuna 2.8.0

Error messages, stack traces, or logs

Error message as the title of the issue: ‘TPESampler’ object has no attribute ‘_group’

/usr/local/lib/python3.7/dist-packages/optuna/study.py in optimize(self, func, n_trials, timeout, n_jobs, catch, callbacks, gc_after_trial, show_progress_bar)
    408             callbacks=callbacks,
    409             gc_after_trial=gc_after_trial,
--> 410             show_progress_bar=show_progress_bar,
    411         )
    412 

/usr/local/lib/python3.7/dist-packages/optuna/_optimize.py in _optimize(study, func, n_trials, timeout, n_jobs, catch, callbacks, gc_after_trial, show_progress_bar)
     73                 reseed_sampler_rng=False,
     74                 time_start=None,
---> 75                 progress_bar=progress_bar,
     76             )
     77         else:

/usr/local/lib/python3.7/dist-packages/optuna/_optimize.py in _optimize_sequential(study, func, n_trials, timeout, catch, callbacks, gc_after_trial, reseed_sampler_rng, time_start, progress_bar)
    160 
    161         try:
--> 162             trial = _run_trial(study, func, catch)
    163         except Exception:
    164             raise

/usr/local/lib/python3.7/dist-packages/optuna/_optimize.py in _run_trial(study, func, catch)
    195                 failed_trial_callback(study, failed_trial)
    196 
--> 197     trial = study.ask()
    198 
    199     state: Optional[TrialState] = None

/usr/local/lib/python3.7/dist-packages/optuna/study.py in ask(self, fixed_distributions)
    485         if trial_id is None:
    486             trial_id = self._storage.create_new_trial(self._study_id)
--> 487         trial = trial_module.Trial(self, trial_id)
    488 
    489         for name, param in fixed_distributions.items():

/usr/local/lib/python3.7/dist-packages/optuna/trial/_trial.py in __init__(self, study, trial_id)
     55         self.storage = self.study._storage
     56 
---> 57         self._init_relative_params()
     58 
     59     def _init_relative_params(self) -> None:

/usr/local/lib/python3.7/dist-packages/optuna/trial/_trial.py in _init_relative_params(self)
     65         self.relative_search_space = self.study.sampler.infer_relative_search_space(study, trial)
     66         self.relative_params = self.study.sampler.sample_relative(
---> 67             study, trial, self.relative_search_space
     68         )
     69 

/usr/local/lib/python3.7/dist-packages/optuna/samplers/_tpe/sampler.py in sample_relative(self, study, trial, search_space)
    327         self._raise_error_if_multi_objective(study)
    328 
--> 329         if self._group:
    330             assert self._search_space_group is not None
    331             params = {}

AttributeError: 'TPESampler' object has no attribute '_group'

[hvy]

Thanks for the confirmation and good that it worked out.

Yes, providing a better error message, or some alternative logic would definitely be helpful. Were you imaging something like introducing checks to __setstate__ to validate the unpickling? It’s otherwise quite difficult to check for AttributeErrors at the Optuna library level. I’d be curious for alternative solutions. The above approach, if it works, would still be difficult to maintain and also somewhat a Python (or pickle, which a part of the standard library) matter, rather than library.

Just as a side note, Optuna doesn’t officially support pickling/unpickling across different versions, and ideally, you’d persist studies via the RDBStorage in which case we provide storage upgrade functionality via the optuna CLI in case underlying incompatibilities are introduced between releases, e.g. optuna storage upgrade --storage sqlite:///example.db.

[hvy]

few hours earlier

We just released v2.8, an hour or two or so ago!

I think the study was saved by the earlier version than the current one, 2.8.0

I’m wondering if with “saved”, you don’t happen to mean pickling the study. If so, I think this is expected behavior. A (private) attribute was introduced to the TPESampler in v2.8 and if you’ve pickled or serialized the sampler before this introduction, you’ll encounter this AttributeError when unpickling it with v2.8.