Split autoformatters and linters into different workflows and CI jobs

Status quo

Currently we mix formatters and linters with pre-commit:

formatters

https://github.com/pytorch/vision/blob/578c1546c328c68e601bfd2e2309ed175ee343d9/.pre-commit-config.yaml#L9-L11

https://github.com/pytorch/vision/blob/578c1546c328c68e601bfd2e2309ed175ee343d9/.pre-commit-config.yaml#L23-L26

linters

https://github.com/pytorch/vision/blob/578c1546c328c68e601bfd2e2309ed175ee343d9/.pre-commit-config.yaml#L5-L8

https://github.com/pytorch/vision/blob/578c1546c328c68e601bfd2e2309ed175ee343d9/.pre-commit-config.yaml#L31-L32

https://github.com/pytorch/vision/blob/578c1546c328c68e601bfd2e2309ed175ee343d9/.pre-commit-config.yaml#L37

In addition we have a separate CI job that lints with mypy

https://github.com/pytorch/vision/blob/578c1546c328c68e601bfd2e2309ed175ee343d9/.circleci/config.yml#L298

Proposal

I propose we change the rationale from the two CI jobs from pre-commit and mypy to code format and lint. That means, we would only keep autoformatters in our pre-commit configuration and move all linters to what is currently the mypy job.

Pros

• We could use pre-commit as “single source of truth” for formatting code. Currently they mentioned as “purely optional” in our contribution guide. Since pre-commit supports running the hooks manually, we can simply treat it as our way to bundle all autoformatters while the user does not need to know or care what exactly is run.
• We could simply add new autoformatters if they are available through pre-commit, e.g. #5158. If anyone applies code formatting through pre-commit anyway, that won’t break any workflow.

Cons

• We would loose the ability to run linters in a bundled manner. I don’t think this is a strong con, since linters such as flake8 or pydocstyle trigger very seldom anyway due to the auto-formatting. Plus, we already need to run mypy separately.

cc @seemethere

[NicolasHug]

flake8 often picks up import problems that black doesnt.

That is not true.

From personal experience the one thing that black doesn’t pick up is xyz imported but not used. This happens often when refactoring (or just writing) code.

I see value in allowing pre-commits, but I wouldn’t make this mandatory either. It’s just sugar on top of the more low-level tools, and I think it’s important to keep them. To answer a question from @pmeier from another thread https://github.com/pytorch/vision/issues/5168#issuecomment-1007263427

Does a contributor need to know for example how to run all linters manually?

IMHO yes, if only for educational purpose. It’s important that they know what’s happening under the hood, for example to resolve errors more efficiently.

[pmeier]

flake8 often picks up import problems that black doesnt.

~That is not true~ (EDIT: I missed the word “import” in @datumbox’s statement. I agree, unused imports is one of the few things that ufmt will not handle). Especially in combination with usort, there are very few things that are left for it to pick up. Quoting from our contribution guidelines:

Similarly, you can check for flake8 errors with flake8 torchvision, although they should be fairly rare considering that most of the errors are automatically taken care of by ufmt already.

It basically comes down to stuff that cannot be fixed automatically, e.g. from foo import *.

I personally don’t use it and I don’t like the idea of having changes automatically done on my code silently without the ability to review them.

That can’t happen. If you have the hooks installed, git commit something, and hook fails, it is as if the git commit has never happened. You will see all changes made by the hooks as unstaged changes that you need to manually stage again. Only if no hook fails, git commit is actually executed.

Besides, just using the pre-commit framework does not mean you have to use the hooks. You can simply do pre-commit run, which will execute all formatters on the staged files. If you don’t pre-commit install, you will have no changes whatsoever to the git behavior. Thus, without installing the hooks, pre-commit basically acts as grouper for all formatters while also handling setting them up in their own environments.