The Configuration

Let’s use this issue to collect thoughts on how the construction of the configuration should work. Like with #663 I suggest we edit this issue to reflect the results of our discussion.

Construction

Ways to define a (default) config:

1. Plain Dict

   ex.add_config({'a': 10, 'b': 'foo'})
   

2. Config Files Essentially the same as plain dict, but from a file.

   ex.add_config_file('my_config.json')
   

3. Programmatically This corresponds to current config scopes. Allows embedding some logic into the configuration process.

   def configure(cfg):
       cfg.a = 10
       cfg.b = 2 * cfg.a
   

4. Gather from Functions Collect configuration from the signatures of certain functions. Inspired by gin-config and gather_default_args.

   @gather_config
   def some_function(a: int = 10, b: str = 'foo'):
   """ Possibly provide argument documentation in the doc-string"""
       ....
   

5. Environment Variables An optional env-key could be specified for the parameters to automatically load them from environment variables. This might require a type specification for converting the value. E.g.:

   def configure(cfg):
       cfg.foo = Parameter(1337, env_key='SACRED_FOO', type=int)
   

Config Structure

The configuration should support:

• dotted access for keys that are valid python identifiers
• arbitrary (even non-str) keys as long as they are hashable builtin types (str, int, float, tuple)
• arbitrary objects as values (e.g. pathlib.Path, datetime, numpy.array, …)

Parameters

A Parameter (Param, Argument, Arg, Entry, Value) class keeps track of the all the meta information related to a configuration value. This includes:

• value
• description
• type
• validation

Serialization

Importantly the configuration has to be serializable to JSON, so that it can be easily stored in a MongoDB. This will require integration of a tool like jsonpickle.

[ngarneau]

I am having a hard time figuring out how to properly design a “fully configurable” experiment. Let me explain my use case:

1. I have some basic configurations. I mostly want to override them.
2. I have some configurations that I want to override from the command line.
3. I have some configurations that I want to update programmatically.

What is the best way to use Sacred? It looks like using automain does not fit properly since I need to rerun exp.run('command') with my programmatically updated configurations. Doing this reruns the basic configs scope thus clears out the configuration that I override in the command line.

I may be missing something here but to my sense, the most global configuration (@exp.config decorator) should be run only once?

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.