Refactoring the config system

Yep, the config system strikes again. I’ll try to keep it short this time. The main point is that, while the current config system works fairly well, it’s quickly become a mess and difficult to maintain. For example, here are just two problems with it:

1. The exported dictionaries config and camera_config are ambiguous – #283
2. You can’t set the config when running from a file (for example when testing or making documentation) – see #293
3. file_writer_config, cameraconfig_, config are confusing, and sometimes read/parse the same cfg files twice – see #443

The plan

My plan for refactoring is the following, modified from this comment:

I want to refactor the config system so that the order of events would be the following:

1. A script executes import manim, which in turn executes manim/__init__.py. This can be a user scene script or our own manim/__main__.py.
2. In __init__.py, the first line of code is import config which executes config/__init__.py.
3. config/__init__.py first executes config/config.py.
4. In config.py, as much as possible of the config dict should be defined. In particular, all config files must be read and parsed. Also, config.py also parses the sections of the configuration corresponding to the logger, and exports them as well. All of config, camera_config, and file_writer_config are defined and exported. NOTE the logger is NOT set here.
5. Back in config/__init__.py, the next line it runs executes config/logger.py.
6. config/logger.py takes the config options exported from config/config.py and uses them to set the logger. NOTE this eliminates the need to execute _run_config twice.
7. Back in config/__init__.py, everything exported by config/config.py and config/logger.py is exported up to the top-level package. After this, the config subpackage should never really have to be used again (other than for importing config/logger). config/__init__.py now returns execution to the top-level __init__.py.
8. Back in __init__.py, here comes the split.

• If the original script is a user script, then we are done and execution is returned to the user. They can then go about defining their Scene classes and ultimately rendering them.
• If the original script was __main__ because manim has been called from the command line, __init__.py returns execution to __main__.main(). Now, this function must deal with parsing command line flags and then updating the global config dict. This can easily be done by implementing e.g. manim.config.parse_cli(args). Note this will remove the problem we’ve had in the past where config.py had to determine whether it had been called from the command line (i.e. the hacky _from_command_line function). Once the CLI flags have been parsed and the config updated, __main__.main() simply decides what to execute next, depending on the subcommand used. Thus, a big chunk of cfg_subcommands will be ported into __main__.py.

Associated PRs

This is a fairly major overhaul so I’ll try to keep the PRs short and self-contained so that they are easy to review.

• #340 (merged): add the tempconfig context manager for testing all of the upcoming changes. Also make Camera read the config at instance construction, not at class definition.

• #376 (merged): creates a new subpackage for all the stuff related to the config system.

• #397 (merged): makes all modules use the config dict as little as possible during library initialization.

• #543 (merged): refactors the config subpackage so that only config/__init__.py defines and exports global variables, whereas all other files just contain utilities.

• #620 (merged): replaces the global config with a custom class ManimConfig.

[leotrs]

Ok I see now. Then tempconfig is not what you want. What you want is to call config.update(some_dict) at the beginning of the pytest session. I’m currently working on this.

[PgBiel]

Fair. I can manually add each config option as an attribute, set them to None, and then replace them with the corresponding property dynamically afterward. I think that’d work?

make sure to specify their types as well, then sure