add trials to study / build study from trials

I would like to add trials to studies or artificially build a study from trials.

The second method might solve https://github.com/optuna/optuna/issues/685 as well

Motivation

In some workflow, you might want to optimize different metrics that are computed from the same results.

For example, in classification problems, people hesitate to maximize the cross-entropy, the accuracy, the F1 score, etc… Similarly, when doing k-fold cross-validation, people might want to optimize the average or the minimum of the fold scores.

In all these cases, it is very cheap to compute all the scores instead of just one, and this data could be provided to avoid cold start in other runs.

Description

I think that providing an argument trials in create_study would be the most practical solution, as it avoids having to reindex a database. It also solves https://github.com/optuna/optuna/issues/685 without having to actually delete trials because users will be able to recreate a study object with a clean list of trials.

[louisabraham]

Sure, it looks very much resolved. I’ll play with them later. Thank you for your hard work!

[hvy]

@louisabraham Two new experimental APIs were introduced in #1335, one for adding trials to a study and one for creating arbitrary trials (this is a rather low-level API). They should address this issue. Have you taken a look at it? I’m wondering if we cannot close this issue as resolved.