Allow adding help messages to command line arguments

🚀 Feature Request

It would be nice to be able to add help messages (similar to the argparse library) to command line arguments defined in .yaml configs / structured configs.

These help messages should be available to view conveniently when typing python myapp.py --help into the command line so that the program’s user doesn’t have to look through the config files and / or the rest of the code to figure out what some command line arguments do.

Motivation

Is your feature request related to a problem? Please describe.

It would be nice if users of programs written with hydra could see helpful messages for command line arguments.

It’s already available as a feature in Python’s argparse library by passing a help parameter containing a help message to parser.add_argument() as demonstrated here.

Adding the ability to add help messages to command line arguments in hydra may help to improve a user’s understanding of what a program does.

Pitch

Describe the solution you’d like

The solution should:

• Allow a developer using Hydra to define a help message for each command line argument and each group defined in .yaml files or structured configs.

• Allow an end user of a program written with Hydra to see what command line arguments can be passed as well as a description associated with each of them.

The solution could also optionally have the following features:

• It would also be nice if the defaults for each attribute were optionally shown to the user in the help message. An argparse equivalent is shown here.

• It would also be nice if the type of each attribute were optionally shown in the help message (if known, such as when using structured configs). An argparse equivalent is shown here.

Describe alternatives you’ve considered

I have a few suggestions on how this could be implemented (1-4 below) in order of increasing plausibility. Suggestions 1-2 have possible solutions for both .yaml files and structured configs. Suggestions 3-4 have better suggestions for structured configs.

I feel like suggestion 4 might be the best for structured configs. I’m not sure if help messages could be defined better in .yaml files.

1. How it could be done currently without any changes

At the moment, I understand that help messages for hydra can be configured as per this tutorial (repeated below):

hydra:
  help:
    # App name, override to match the name your app is known by
    app_name: ${hydra.job.name}

    # Help header, customize to describe your app to your users
    header: |
      ${hydra.help.app_name} is powered by Hydra.

    footer: |
      Powered by Hydra (https://hydra.cc)
      Use --hydra-help to view Hydra specific help

    # Basic Hydra flags:
    #   $FLAGS_HELP
    #
    # Config groups, choose one of:
    #   $APP_CONFIG_GROUPS: All config groups that does not start with hydra/.
    #   $HYDRA_CONFIG_GROUPS: All the Hydra config groups (starts with hydra/)
    #
    # Configuration generated with overrides:
    #   $CONFIG : Generated config
    #
    template: |
      ${hydra.help.header}
      == Configuration groups ==
      Compose your configuration from those groups (group=option)

      $APP_CONFIG_GROUPS

      == Config ==
      Override anything in the config (foo.bar=value)

      $CONFIG

      ${hydra.help.footer}

e.g. It’s possible to add help messages in manually by overriding hydra.help.template as needed. This could be done by manually typing the desired result of python my_app.py --help. But, this seems counterproductive (e.g. what if something is updated in db.mysql in the tutorial’s example? It’s possible to easily forget to update any manual help messages in hydra.help.template!).

Instead, perhaps help messages could be defined at the same time and place as the associated config attributes. So, here’s my next alternative suggestion:

2. Parse comments in .yaml files / structured configs

As inspiration for this alternative example, I happened to come across the next Hydra version’s doc site. I found a section on the “Nevergrad Sweeper Plugin” example here.

The comments in the example (kind of) felt like I could be reading the help text that would appear for this config when using python my_app.py --help. So, a potential quick workaround could be to parse comments directly above a config attribute as help text.

Example of using this for a config.yaml file (adapted from the Hydra tutorial website):

# Group to decide what type of database to use
db:
  # Driver to use for the database
  driver: mysql
  # Username for the database
  user: omry
  # Password for the database
  pass: secret

Example of using this for structured configs (adapted from the Hydra tutorial website):

@dataclass
class MySQLConfig:
    # Driver to use for the database
    driver: str = "mysql"
    # Username for the database
    user: str = "omry"
    # Password for the database
    pass: str = "secret"

@dataclass
class Config(DictConfig):
    # Group to decide what type of database to use
    db: MySQLConfig = MISSING

When a user types python myapp.py --help, I’d expect the following help message to appear (adapted from the Hydra tutorial website):

my_app is powered by Hydra.

== Configuration groups ==
Compose your configuration from those groups (group=option)

db: mysql - Group to decide what type of database to use


== Config ==
Override anything in the config (foo.bar=value)

db:                                 - Group to decide what type of database to use
  driver: mysql                - Driver to use for the database
  user: omry                   - Username for the database
  pass: secret                  - Password for the database


Powered by Hydra (https://hydra.cc)
Use --hydra-help to view Hydra specific help

(Sorry the indentation is all weird on the help text - this seems to be a “feature” of GitHub)

This method, although easy to implement, feels very “hacky”. Especially for the structured configs as it’s just Python comments that I’ve used in this alternative suggestion.

3. Define a @dataclass helper function

Perhaps, in the case of structured configs at least, a helper function could be used to define help text. e.g.:

@dataclass
class MySQLConfig:
    driver: str = "mysql"
    user: str = "omry"
    pass: str = "secret"

    def help(self) -> Dict[str, str]:
        helpDict = {
                            "driver": "Driver to use for the database",
                            "user": "Username for the database",
                            "pass": "Password for the database"
        }
        return helpDict

@dataclass
class Config(DictConfig):
    db: MySQLConfig = MISSING

    def help(self) -> Dict[str, str]:
        helpDict = {"db": "Group to decide what type of database to use"}
        return helpDict

This could work if you define that help() should be a method defined by structured configs to add help messages.

However, this has the disadvantage that if the structured config class is large, then it would get hard to check if every attribute that you want to have a help message has a help message in helpDict.

This should output the same help message as in the previous suggestion.

4. Using @dataclass’s metadata parameter

The dataclasses.field() method has a metadata parameter. The definition provided by the Python docs is:

metadata: This can be a mapping or None. None is treated as an empty dict. This value is wrapped in MappingProxyType() to make it read-only, and exposed on the Field object. It is not used at all by Data Classes, and is provided as a third-party extension mechanism. Multiple third-parties can each have their own key, to use as a namespace in the metadata.

This seems like the best solution to implement (for structured configs at least) for the following reasons:

• metadata is read-only

• Each attribute’s help message is defined at the location the attribute is defined which makes it easy to check if a help message has been written for a given attribute.

• “metadata is provided as a third-party extension mechanism” - This is exactly the use case!

• It’s less lines of code compared to my other structured config examples (see below)

An example of how this could be used is to suggest to a Hydra developer to use the keyword "help" as a metadata dictionary key to a help message for each attribute that the developer wants to assign a help message.

@dataclass
class MySQLConfig:
    driver: str = field(default="mysql", metadata={"help", "Driver to use for the database"})
    user: str = field(default="omry", metadata={"help", "Username for the database"})
    pass: str = field(default="secret", metadata={"help", "Password for the database"})

@dataclass
class Config(DictConfig):
    db: MySQLConfig = field(default=MISSING, metadata={"help", "Group to decide what type of database to use"})

Again, this should output the same help message as in the previous suggestion.

I’m sorry that this feature request is so long, I just kept thinking of alternative ways the solution could possibly be implemented.

That being said, of course, none of these suggested implementations are necessarily the best - these are just suggestions! 😄

If something’s not clear please let me know!

Are you willing to open a pull request? (See CONTRIBUTING)

No, sorry. 😖

Additional context

Add any other context or screenshots about the feature request here.

[SigmaX]

Just chiming in with support for this feature. Help messages & a self-documenting CLI are my favorite feature of click & argparse, so the lack of them is a source of hesitation when considering Hydra for projects (along with the inability to use Hydra and click/argparse together in the same program #409 , which would be a reasonable compromise in many situations).

[oliver-batchelor]

simple-parsing - https://github.com/lebrice/SimpleParsing (an argparse library using dataclass) supports help well.

It uses both docstring and field comments like this:

@dataclass
class Foo:
     """ This is some help for the whole of Foo"""
    someOption : Bool = False   # This turns the option on
    blah : str # This is some more help for the blah