WandB client error when tuple used in sweeps config

Python version: 3.7 Wandb versions tested: 0.10.6 to 0.10.10 OS: Linux

Summary: Wandb sweep agents will run once and then error out on all subsequent runs. The observed error is the anaconda 400 / graphql 500 error that I’ve seen here frequently, but the nested error is the common numpy error that goes something like “boolean from array makes no sense, use np.all or np.any”. It was pretty uninformative, but I think I’ve tracked down the issue.

Use case: I have a couple hyperparameters that are categorical but dependent on one another. Example: param A could be values X, Y, Z and param B could be values L, J, K, but only certain combos make sense, e.g., (X, L), (X, J), (Y, J), (Z, K). Rather having those listed as separate params and wandb passing combos that don’t make sense, I configured those params to be tuples. Note that the current docs say categorical values can have distribution defined by “values: [(value), (value)…]”, but does not suggest that “value” cannot be a tuple.

Sample configuration dictionary:

        "parameters": {
            ...
            "linked_hypers": {"distribution": "categorical", "values": [(0, 0), (8, 8), (8, 16), (16, 8), (16, 16)]},
            ...
        },

Solution: I ended up encoding these tuples as strs in the config, e.g., “8,8”, then splitting and casting to the actual parameter values in my code. The errors stopped. However, it’d be nice to either have support for tuples, have informative error messages when this happens, or have more info in the docs, but I’m guessing the quickest/easiest solution on your end is just to have more rigorous input checks when sweep configs are submitted.

(Apologies for not grabbing the actual stacktrace)

Thanks!

[vanpelt]

@nsfabina we’re going to support tuples in the next release of our client library due out later this week. We’ll update this ticket once it’s released.

[farizrahman4u]

Got a script to reproduce? Following works for me:

import wandb

sweep_config = {
  "name": "My Sweep",
  "method": "grid",
  "parameters": {
        "param1": {
            "values": [1, 2, 3]
        },
        "param2":{
            "distribution": "categorical",
            "values": [(0, 0), (8, 8), (8, 16), (16, 8), (16, 16)]
        }
    }
}

sweep_id = wandb.sweep(sweep_config)

def train():
    run = wandb.init()
    print(run.config.param2)
    run.finish()

sweep_id = wandb.sweep(sweep_config)
agent = wandb.agent(sweep_id=sweep_id, function=train)
agent.run()