False positives using when using pytest-xdist -f (--looponfail)
See original GitHub issueWhen doing TDD, after the first mypy error, the plugin start emitting errors on collection.
It will continue normally until you hit an actual mypy fail, then in the follow round (after mypy error is fixed) this warning is emitted forever until pytest is killed and restarted.
Steps to Reproduce (assumes python 3.8 and pipenv available):
mkdir ~/tmp20200412
cd ~/tmp20200412
git clone git@bitbucket.org:y2kbugger/test_driven_development_by_example_tdd_follow_along.git
cd ./test*
git checkout b0a2f56
- replicate versions exactly from Pipfile.lock
pipenv sync
- put venv on PATH
pipenv shell
- run pytest
pytest --mypy -f money.py test_money.py
In another terminal:
cd ~/tmp20200412/test*
- cause mypy error
echo 'lol: str = 2' >> money.py
- undo mypy error
git checkout b0a2f56 -- .
###################################### waiting for changes #######################################
### Watching: /home/y2k/devel/test_driven_development_by_example_tdd_follow_along
# MODIFIED /home/y2k/devel/test_driven_development_by_example_tdd_follow_along/money.py
============================= test session starts ==============================
platform linux -- Python 3.8.2, pytest-5.4.1, py-1.8.1, pluggy-0.13.1
rootdir: /home/y2k/devel/test_driven_development_by_example_tdd_follow_along
plugins: forked-1.1.3, mypy-0.6.1, xdist-1.31.0, flake8-1.0.4
collected 0 items / 1 error
collected 0 items / 1 error
==================================== ERRORS ====================================
________________________ ERROR collecting test session _________________________
../../.local/share/virtualenvs/test_driven_development_by_example_tdd_fol-TPMT2dI5/lib/python3.8/site-packages/_pytest/runner.py:244: in from_call
result = func()
../../.local/share/virtualenvs/test_driven_development_by_example_tdd_fol-TPMT2dI5/lib/python3.8/site-packages/_pytest/runner.py:264: in <lambda>
call = CallInfo.from_call(lambda: list(collector.collect()), "collect")
../../.local/share/virtualenvs/test_driven_development_by_example_tdd_fol-TPMT2dI5/lib/python3.8/site-packages/_pytest/main.py:498: in collect
yield from self._collect(fspath, parts)
../../.local/share/virtualenvs/test_driven_development_by_example_tdd_fol-TPMT2dI5/lib/python3.8/site-packages/_pytest/main.py:537: in _collect
assert not names, "invalid arg {!r}".format((argpath, names))
E AssertionError: invalid arg (local('/home/y2k/devel/test_driven_development_by_example_tdd_follow_along'), ['mypy'])
=========================== short test summary info ============================
ERROR - AssertionError: invalid arg (local('/home/y2k/devel/test_driven_deve...
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
=============================== 1 error in 0.22s ===============================
######################################### LOOPONFAILING ##########################################
money.py::mypy
test_money.py::test_multiplication
test_money.py::test_franc_multiplication
test_money.py::test_equality
test_money.py::test_currency
::mypy
###################################### waiting for changes #######################################
### Watching: /home/y2k/devel/test_driven_development_by_example_tdd_follow_along
Note I kill with keyboard interrupt here:
^C
Then everything goes fine again:
(test_driven_development_by_example_tdd_follow_along) [y2k@15r test_driven_development_by_example_tdd_follow_along]
$ pytest --mypy -f money.py test_money.py
============================= test session starts ==============================
platform linux -- Python 3.8.2, pytest-5.4.1, py-1.8.1, pluggy-0.13.1
rootdir: /home/y2k/devel/test_driven_development_by_example_tdd_follow_along
plugins: forked-1.1.3, mypy-0.6.1, xdist-1.31.0, flake8-1.0.4
collected 7 items
collected 7 items
money.py . [ 12%]
test_money.py ...... [ 87%]
. . [100%]. [100%]
===================================== mypy =====================================
Success: no issues found in 2 source files
============================== 9 passed in 0.34s ===============================
###################################### waiting for changes #######################################
### Watching: /home/y2k/devel/test_driven_development_by_example_tdd_follow_along
Issue Analytics
- State:
- Created 3 years ago
- Reactions:2
- Comments:6
Top Results From Across the Web
False positives using when using pytest-xdist -f (--looponfail)
When doing TDD, after the first mypy error, the plugin start emitting errors on collection. It will continue normally until you hit an ......
Read more >Loop on fail - Small Pytest Trick to Boost Your Productivity
-f, --looponfail run tests in subprocess, wait for modified files and re-run failing test set until all pass. Amazing! especially if you have ......
Read more >pytest Documentation - Read the Docs
When distributing tests on the local machine using pytest-xdist, ... This avoids false positives caused by limited floating-point precision, ...
Read more >What Is a False Positive COVID-19 Test? - Prevention.com
Therefore, “false positive” means that you have been delivered a positive result, but are not actually infected with the SARS-CoV-2 virus. How ...
Read more >Changelog — pytest documentation
issue #5971: Fix a pytest-xdist crash when dealing with exceptions raised in subprocesses created by the multiprocessing module.
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
Would you mind cutting a release with this? I would love to have this working.
When a test fails,
pytest-xdist
holds onto the node ID of the failing test. TheMypyStatusItem
that is injected inpytest_collection_modifyitems
hasparent=session
which produces a node ID of::mypy
(wheremypy
isnodeid_name
). Running that node ID explicitly (likexdist
does) reproduces this: