[Launcher: Local parallel sweep]

🚀 Feature Request

Is it doable to execute sweeps locally in parallel (with a variable like ntasks_per_node in config.yaml)? Additionally, being able to specify a list of gpu indices and run the sweep in parallel on those (relying on the env var CUDA_VISIBLE_DEVICES) could be useful.

Motivation

Is your feature request related to a problem? Please describe. The motivation is that without a slurm system configured, a simple parallel launcher would allow easier and faster computations than executing several times the same command.

Pitch

Describe the solution you’d like A launcher object allowing a sweep in parallel.

Describe alternatives you’ve considered I tried adapting the BasicLauncher using joblib but did not succeed so far. One issue being that I get a Invalid plugin error because my launcher class does start neither with hydra_plugins nor with hydra._internal.core_plugins..

Additional context

Thanks fo the great library! 😃

[omry]

The new plugin is published. It supports Hydra 1.0.0 which is not yet released. you can try it by checkout out Hydra from master and installing the plugin with pip install.

Plugin website page.

[emilemathieutmp]

Thanks @omry for reaching back 😃

I eventually succeeded in implementing a parallel Launcher based on joblib (can be found here)

I still have a few issues:

• How to avoid the Invalid plugin error because my launcher class does start neither with hydra_plugins nor with hydra._internal.core_plugins. ?
• I don’t know why but the OmegaConf cannot resolve an inter_type now at some point which I hard fixed with something like value = datetime.datetime.now().strftime(inter_key).

If you have any hint that’d be great! 😃