Passing allow_val_change=True to wandb.init() prevents sweeps from changing hyperparameters

I’m not sure if this is strictly a bug, but it’s certainly not what I would expect.

• Weights and Biases version: 0.9.3
• Python version: 3.7.6
• Operating System: Linux

Description

Passing allow_val_change=True to wandb.init() clamps the config values to their defaults when running a sweep, and thus no new values are used. The command line inputs introduced by wandb seem to be ignored.

What I Did

I did the usual of defining sweep.yaml, running wandb sweep sweep.yaml and wand agent <agent_id>. In my train.py script, I print wandb.config after the wandb.init() call, which confirms that the hyperparameters in the sweep are always fixed to the defualt config values that I pass to wandb.init(). Setting allow_val_change=False prevents this unexpected behaviour.

I’d expect allow_val_change to simply allow me to overwrite wandb.config values in my training script, but still allow sweeps to run and modulate the hyperparameters.

P.S. Thank you very much for making this fantastic package; I’m finally emerging from my cave where I manually tune hyperparameters like a maniac.

[ariG23498]

Hey @tom-andersson I tried reproducing the code. I found a simple synatx error in your code snippet otherwise the code seems to be doing what is expected.

Train.py

import wandb
defaults = dict(seed=42, filter_size=3)
wandb.init(entity="xxx", allow_val_change=True, project="xxx",config=defaults)
print('\n\nHyperparams:')
print(wandb.config)
print('\n\n')
log = 2*wandb.config["filter_size"] - 0.01*wandb.config["seed"]
wandb.log({"output":log})
wandb.finish()

Sweep.yml

project: xxx
entity: xxx
program: train.py
method: bayes
metric:
  goal: minimize
  name: output
parameters:
  seed:
    distribution: int_uniform
    max: 100
    min: 42
  filter_size:
    distribution: int_uniform
    max: 7
    min: 3

Code error

wandb.init(project="my_project_name", allow_val_changes=True, config=defaults)

The parameter should have been allow_val_change instead of allow_val_changes.

When I use the allow_val_change=True the print statements seem to be printing out the different paramter values as should be

We are closing this ticket as the solution is found. Please let us know if the problem still persists. 😄

[issue-label-bot[bot]]

Issue-Label Bot is automatically applying the label bug to this issue, with a confidence of 0.73. Please mark this comment with 👍 or 👎 to give our bot feedback!

Links: app homepage, dashboard and code for this bot.