The new impl for CONFIG_MAPPING prevents users from adding any custom models

Environment info

• transformers version: 4.10+
• Platform: Ubuntu 18.04
• Python version: 3.7.11
• PyTorch version (GPU?): N/A
• Tensorflow version (GPU?): N/A
• Using GPU in script?: N/A
• Using distributed or parallel set-up in script?: No.

Who can help

Information

Model I am using (Bert, XLNet …): Custom model

The problem arises when using:

• the official example scripts: (give details below)
• my own modified scripts: (give details below)

The tasks I am working on is:

• an official GLUE/SQUaD task: (give the name)
• my own task or dataset: (give details below)

To reproduce

See: https://github.com/huggingface/transformers/blob/010965dcde8ce9526f6a7e6e2c3f36276c153708/src/transformers/models/auto/configuration_auto.py#L297

This was changed from the design in version 4.9 which used an OrderedDict instead of the new _LazyConfigMapping. The current design makes it so users cannot add their own custom models by assigning names and classes to the following registries (example: classification tasks):

• CONFIG_MAPPING in transformers.models.auto.configuration_auto, and
• MODEL_FOR_SEQUENCE_CLASSIFICATION_MAPPING in transformers.models.auto.modeling_auto.

Expected behavior

Either a mechanism to add custom Configs (and the corresponding models) with documentation for it, or documentation for whatever other recommended method. Possibly that already exists, but I haven’t found it yet.

@sgugger

[LysandreJik]

Hello @aleSuglia, @sgugger is working on that API here: https://github.com/huggingface/transformers/pull/13989

It should land in the next few days.

[aleSuglia]

@sgugger @LysandreJik any updates on this? Thanks!