Hydra/OmegaConf configuration structure
See original GitHub issue🚀 Feature Request
Hydra provides interesting features to create hierarchical configuration by composition and override it through config files and the command line. I think that this ability will be useful for catalyst pipelines with configuration file. Now one configuration file contains all pipeline settings inside, but if we split it to for example model_params/data_params/runner_params/and so on then we will be able to get configuration structure distributed in sence. This will allow us to quickly run experiments with different combinations of settings and also change settings in the command line
Example pipeline:
pipeline/
conf/
config.yaml
model/
resnet.yaml
mobilenet.yaml
efficientnet.yaml
data/
imagenet.yaml
coco.yaml
runner/
runner1.yaml
runner2.yaml
src/
callbacks.py
model.py
...
Run example:
catalyst-dl run config=conf/config.yaml model=resnet data=imagenet runner=runner1
Possible solution
- Create class HConfigExperiment class
- Change catalyst-dl run script
Motivation
Combining configuration parts for different experiments. Change settings in command line.
Proposal
Alternatives
Additional context
I’ve already written simple MNIST classification pipeline with hydra configuration structure. But I can run it only with python train.py
command.
Issue Analytics
- State:
- Created 3 years ago
- Comments:10 (6 by maintainers)
@Scitator Ok. Moving to slack now.
@Scitator They are almost the same but with Hydra/OmegaConf you can also:
catalyst-dl run config=conf/config.yaml model=resnet data=imagenet runner=runner1 model.resnet=50/101/...
One more advantage of using Hydra/OmegaConf is that Hydra can say ‘Catalyst uses us’ and Catalyst can also say ‘We use Hydra’. So the community will grow. And we will be happier =)