question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Bayesianoptimization algorithm does not work when hyperparameter are negative doubles

See original GitHub issue

/kind bug

What steps did you take and what happened:

Katib suggest NaN values when submitting an experiment with:

  • a parameter with parametertype: double
  • a feasibleSpace including negative numbers
  • bayesianoptimization as algorithm

For example this kind of yaml makes Katib suggesting NaN for the parameter my-param.

apiVersion: "kubeflow.org/v1alpha3"
kind: Experiment
metadata:
  namespace: kubeflow
  labels:
    controller-tools.k8s.io: "1.0"
  name: my-buggy-experiment
spec:
  objective:
    type: maximize
    goal: 0.99
    objectiveMetricName: my_metric
  algorithm:
    algorithmName: bayesianoptimization
    algorithmSettings:
      - name: "n_initial_points"
        value: "5"
  metricsCollectorSpec:
    collector:
      kind: StdOut
  parallelTrialCount: 1
  maxTrialCount: 2
  maxFailedTrialCount: 3
  parameters:
    - name: --my-param
      parameterType: double
      feasibleSpace:
        min: "-1.0"
        max: "1.0"
trialTemplate:
    goTemplate:
        rawTemplate: |-
          apiVersion: batch/v1
          kind: Job
          metadata:
            name: {{.Trial}}
            namespace: {{.NameSpace}}
          spec:
            template:
              spec:
                containers:
                - name: {{.Trial}}
                  image: some_image
                  command:
                  - "python"
                  - "some_script.py"
                  {{- with .HyperParameters}}
                  {{- range .}}
                  - "{{.Name}}={{.Value}}"
                  {{- end}}
                  {{- end}}
                restartPolicy: Never

What did you expect to happen: Katib should not suggest NaN for my-param but values between -1 and 1.

Anything else you would like to add: Is it due to the log-uniform prior in https://github.com/kubeflow/katib/blob/489d52b0094e36ccb53d26cf0127add9e00bd606/pkg/suggestion/v1alpha3/skopt/base_skopt_service.py line 43?

Environment:

  • Kubeflow version: 0.7.0
  • Minikube version:
  • Kubernetes version: (use kubectl version): 1.14.0
  • OS (e.g. from /etc/os-release): Ubuntu 18.04.3 LTS

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Reactions:1
  • Comments:7 (4 by maintainers)

github_iconTop GitHub Comments

1reaction
issue-label-bot[bot]commented, Jan 30, 2020

Issue-Label Bot is automatically applying the labels:

Label Probability
bug 0.99

Please mark this comment with 👍 or 👎 to give our bot feedback! Links: app homepage, dashboard and code for this bot.

0reactions
andreyvelichcommented, Mar 18, 2021

That should be fixed as part of: https://github.com/kubeflow/katib/issues/1207.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Hyperparameter Optimization Techniques for Designing ...
Bayesian Optimization belongs to the Sequential Model-based Optimization methods (SMBO), since it does not optimize the objective function ...
Read more >
MATLAB bayesopt - MathWorks
Negative values mean that the model probably does not error, positive values mean that it probably does error. The model is:
Read more >
Freeze-Thaw Bayesian Optimization - arXiv
Hyperparameter tuning is an important and ubiquitous problem in machine learning that can drastically affect the performance of a model. Given a ...
Read more >
Hyperparameter Tuning with AI Platform | by Imran us Salam
A machine learning algorithm cannot work without a proper set of hyperparameters. And the problem is we have to find it ourselves.
Read more >
A Conceptual Explanation of Bayesian Hyperparameter ...
The problem with hyperparameter optimization is that evaluating the ... At a high-level, Bayesian optimization methods are efficient because ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found