Hydra/OmegaConf configuration structure

🚀 Feature Request

Hydra provides interesting features to create hierarchical configuration by composition and override it through config files and the command line. I think that this ability will be useful for catalyst pipelines with configuration file. Now one configuration file contains all pipeline settings inside, but if we split it to for example model_params/data_params/runner_params/and so on then we will be able to get configuration structure distributed in sence. This will allow us to quickly run experiments with different combinations of settings and also change settings in the command line

Example pipeline:

pipeline/
    conf/
        config.yaml
        model/
            resnet.yaml
            mobilenet.yaml
            efficientnet.yaml
        data/
            imagenet.yaml
            coco.yaml
        runner/
            runner1.yaml
            runner2.yaml
    src/
        callbacks.py
        model.py
        ...

Run example: catalyst-dl run config=conf/config.yaml model=resnet data=imagenet runner=runner1

Possible solution

1. Create class HConfigExperiment class
2. Change catalyst-dl run script

Motivation

Combining configuration parts for different experiments. Change settings in command line.

Proposal

Alternatives

Additional context

I’ve already written simple MNIST classification pipeline with hydra configuration structure. But I can run it only with python train.py command.

[otherman16]

@Scitator Ok. Moving to slack now.

[otherman16]

@Scitator They are almost the same but with Hydra/OmegaConf you can also:

1. Change specific settings in command line. Example: catalyst-dl run config=conf/config.yaml model=resnet data=imagenet runner=runner1 model.resnet=50/101/...
2. Command line Tab Completion
3. Select default configuration
4. As say @omry Structured Configs + Object Instantiating
5. Multi-run different configurations with one command
6. And other features (I don’t know hydra well enough).

One more advantage of using Hydra/OmegaConf is that Hydra can say ‘Catalyst uses us’ and Catalyst can also say ‘We use Hydra’. So the community will grow. And we will be happier =)