Bayesianoptimization algorithm does not work when hyperparameter are negative doubles
See original GitHub issue/kind bug
What steps did you take and what happened:
Katib suggest NaN values when submitting an experiment with:
- a parameter with
parametertype: double
- a feasibleSpace including negative numbers
- bayesianoptimization as algorithm
For example this kind of yaml makes Katib suggesting NaN for the parameter my-param
.
apiVersion: "kubeflow.org/v1alpha3"
kind: Experiment
metadata:
namespace: kubeflow
labels:
controller-tools.k8s.io: "1.0"
name: my-buggy-experiment
spec:
objective:
type: maximize
goal: 0.99
objectiveMetricName: my_metric
algorithm:
algorithmName: bayesianoptimization
algorithmSettings:
- name: "n_initial_points"
value: "5"
metricsCollectorSpec:
collector:
kind: StdOut
parallelTrialCount: 1
maxTrialCount: 2
maxFailedTrialCount: 3
parameters:
- name: --my-param
parameterType: double
feasibleSpace:
min: "-1.0"
max: "1.0"
trialTemplate:
goTemplate:
rawTemplate: |-
apiVersion: batch/v1
kind: Job
metadata:
name: {{.Trial}}
namespace: {{.NameSpace}}
spec:
template:
spec:
containers:
- name: {{.Trial}}
image: some_image
command:
- "python"
- "some_script.py"
{{- with .HyperParameters}}
{{- range .}}
- "{{.Name}}={{.Value}}"
{{- end}}
{{- end}}
restartPolicy: Never
What did you expect to happen:
Katib should not suggest NaN for my-param
but values between -1 and 1.
Anything else you would like to add:
Is it due to the log-uniform
prior in https://github.com/kubeflow/katib/blob/489d52b0094e36ccb53d26cf0127add9e00bd606/pkg/suggestion/v1alpha3/skopt/base_skopt_service.py line 43?
Environment:
- Kubeflow version: 0.7.0
- Minikube version:
- Kubernetes version: (use
kubectl version
): 1.14.0 - OS (e.g. from
/etc/os-release
): Ubuntu 18.04.3 LTS
Issue Analytics
- State:
- Created 4 years ago
- Reactions:1
- Comments:7 (4 by maintainers)
Top Results From Across the Web
Hyperparameter Optimization Techniques for Designing ...
Bayesian Optimization belongs to the Sequential Model-based Optimization methods (SMBO), since it does not optimize the objective function ...
Read more >MATLAB bayesopt - MathWorks
Negative values mean that the model probably does not error, positive values mean that it probably does error. The model is:
Read more >Freeze-Thaw Bayesian Optimization - arXiv
Hyperparameter tuning is an important and ubiquitous problem in machine learning that can drastically affect the performance of a model. Given a ...
Read more >Hyperparameter Tuning with AI Platform | by Imran us Salam
A machine learning algorithm cannot work without a proper set of hyperparameters. And the problem is we have to find it ourselves.
Read more >A Conceptual Explanation of Bayesian Hyperparameter ...
The problem with hyperparameter optimization is that evaluating the ... At a high-level, Bayesian optimization methods are efficient because ...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
Issue-Label Bot is automatically applying the labels:
Please mark this comment with 👍 or 👎 to give our bot feedback! Links: app homepage, dashboard and code for this bot.
That should be fixed as part of: https://github.com/kubeflow/katib/issues/1207.