Transformers crashes mypy

Ubuntu 21.04, tested on both Python 3.8 and Python 3.9 with mypy 0.910

I get a crash caused by transformers 4.10.0 when typechecking a private codebase which imports transformers. This is the error message:

src/transformers/trainer.py:1435: error: INTERNAL ERROR -- Please try using mypy master on Github:
https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
Please report a bug at https://github.com/python/mypy/issues
version: 0.910

(I also get the crash trying mypy master. I’m going to submit a corresponding bug report directly to the mypy issue tracker.)

This is the corresponding line:

https://github.com/huggingface/transformers/blob/b91e65afe0f467e24183928bf57d92b2cef4b69f/src/transformers/trainer.py#L1435

That method is imported into the class, which presumably upsets mypy somehow.

Repro

I can reproduce this locally in the following way:

1. Clone the transformers repo
2. Create a blank virtualenv and install mypy 0.910
3. Add the following mypy.ini file:

   [mypy]
   check_untyped_defs=True
   

4. Run mypy src/transformers/trainer.py

Confusingly, this local repro method also crashes mypy when using the 4.9.1 tag, but I don’t experience the crash in my private codebase when using that transformers version 🤷

cc @sgugger

[aphedges]

Although I hope mypy fixes the bug on their side, I did some digging into possible workarounds:

• The transformers commit that causes mypy to crash is d8fb278, which makes sense. Adding a py.typed enabled the type checking, which exposed the bug even though no code was changed in the commit itself. Any commit before it works without issues.
• If you want to use mypy but are okay with not having transformers type checked, add the following code to mypy.ini:

[mypy-transformers.*]
follow_imports = skip

Hope this helps!

[phiresky]

Just want to mention that the workaround for the newer pyproject.toml-based config files is

[[tool.mypy.overrides]]
follow_imports = "skip"
module = [
  "transformers.*",
]