question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Add option to set determinism with bundle configs

See original GitHub issue

Is your feature request related to a problem? Please describe. There should be a configuration option to enable deterministic training with monai bundle. Currently, a user has to add python code in the “training” portion of the config:

    "training": [
        "$monai.utils.set_determinism(seed=123)",
        "$setattr(torch.backends.cudnn, 'benchmark', True)",
        "$@train#trainer.run()"
    ]

This can be difficult to use in security contrainted environments such as in federated learning where one might want to disable the python code execution in the bundle.

Describe the solution you’d like There should be an option to set the deterministic behavior using configuration style options, e.g., the below to achieve the same behavior as above:

  "determinism": {
    "random_seed": 123,
    "benchmark": True
  }

Describe alternatives you’ve considered MonaiAlgo for executing FL training does not run the “training” section in the config. Therefore the code there is not being executed. One can achieve determinism by using the “required” option for the SupervisedTrainer but this will not work if python code is disabled.

Additional context Especially important feature to support FL.

Issue Analytics

  • State:open
  • Created a year ago
  • Comments:8 (6 by maintainers)

github_iconTop GitHub Comments

1reaction
wylicommented, Aug 22, 2022

maybe one valid feature request is to create two types of permissions for the run_eval flag, https://github.com/Project-MONAI/MONAI/blob/1941283b1862e3f578ccfeb2bdfa864171125029/monai/bundle/config_item.py#L318

  1. whether to eval any user-provided command strings
  2. whether to eval a set of predefined command strings from trusted sources

so that when option 1 is disabled at a system level, the user can still have the flexibility to use the 2nd option to run some predefined commonly used commands.

1reaction
wylicommented, Aug 19, 2022

Yes, perhaps we have this hardcoded in monaialgo for FL, and later extending this to the primary bundle parser if we receive feature requests? cc @Nic-Ma

Read more comments on GitHub >

github_iconTop Results From Across the Web

bundle-config - - Set bundler configuration options
Executing bundle with the BUNDLE_IGNORE_CONFIG environment variable set will cause it to ignore all configuration. Remembering Options. Flags passed to bundle ...
Read more >
BuildAssetBundleOptions.DeterministicAssetBundle
//This script creates a new Menu named “Build Asset” and new options within the menu named “Normal” and “Deterministic Asset Bundle”.
Read more >
Deterministic Asset Bundle Building - Unity Forum
Is there any way to work around this, or some setting perhaps that might need to ... and only add Update bundles into...
Read more >
Configuring bundler using bundle config - BigBinary Blog
To set the value of the configuration setting, use bundle config with name and value. Configuration will be stored in ~/.bundle/config.
Read more >
nix bundle - NixOS
Disable substituters and consider all previously downloaded files up-to-date. --option name value. Set the Nix configuration setting name to value (overriding ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found