Support multi-objective optimization

/kind feature

Describe the solution you’d like

• katib currently use only single-objective optimization, (even though katib could display additional metrics, the optimizing process focused on single objective function)

• However, as you might know well, optuna support multi-objective optimization

  • which means, for example, we can optimize the hyperparameters to maximize accuracy and minimize time-cost simultaneously.

• In such case, I know we can define a new objective function which combines two objectives, such as maximize accuracy / time-cost or something else more complicated.

• However, I wonder is there any plan to support such feature natively in katib.

[g-votte]

Hi, I’m one of the maintainers of Optuna. Thank you for referring to Optuna as a possible backend of the multi-objective API.

As @c-bata mentions, Optuna supports several algorithms for multi-objective optimization, with almost the same interface as single objective optimization.

I think we can go forward by (1) implementing the Optuna suggestion service, and (2) fixing the interface of Katib’s multi-objective functionality.

For (1), I created the PR #1613 to add the suggestion service. I appreciate your taking a look and providing any feedback.

[andreyvelich]

Thank you for creating this @anencore94. I think that feature will require some API modification since type, goal and objectiveMetricName can be unique: https://github.com/kubeflow/katib/blob/master/pkg/apis/controller/common/v1beta1/common_types.go#L94-L103.

@anencore94 @gaocegege @johnugeorge Any ideas how we can implement this ?

@c-bata Does Goptuna support Multi-object optimization ?