check hangs forever when parallel execution is on

Hello. There is the following issue with parallel execution.

Suppose there are two files with unused imports:

test_parallel/test1.py

import sys

test_parallel/test2.py

import sys

When run a check executed in parallel (-j 0 by default), it detects the unused imports, but then it hangs forever. And it needs keyboard interrupt to exit:

$ autoflake test_parallel --check

test_parallel/test1.py: Unused imports/variables detected
test_parallel/test2.py: Unused imports/variables detected
^CProcess SpawnPoolWorker-10:
Process SpawnPoolWorker-9:
Process SpawnPoolWorker-8:
Process SpawnPoolWorker-6:
Process SpawnPoolWorker-5:
Process SpawnPoolWorker-4:
Process SpawnPoolWorker-2:
Process SpawnPoolWorker-1:
Traceback (most recent call last):

Once child process exits after errors found, the main process just sits there waiting for something to be returned from the child process. But child have already exited, so there is nothing to return resulting in a forever wait.

[fsouza]

Ok, the problem is that we’re calling sys.exit and that blocks the pool. #133 fixes it.

[fsouza]

I can repro with the following steps:

％ mkdir test_parallel
％ echo "import sys" > test_parallel/file1.py
％ cp test_parallel/file{1,2}.py
％ autoflake --check test_parallel/*
test_parallel/file2.py: Unused imports/variables detected
test_parallel/file1.py: Unused imports/variables detected
# hangs

Investigating it now, if I don’t succeed I’ll update with more info for others.