pytest-cov 2.8.0 --cov-fail-under reports 0% coverage on Python 3

This was working well. up to v2.7.1. My actual code coverage is 82%, which is accurately reported when I don’t include --cov-fail-under Command: pytest tests/unit -v --cov mypkg --cov-report=html -r s -n 4 --cov-fail-under 80 --junit-xml=unit_test_results.xml

Error: FAIL Required test coverage of 80% not reached. Total coverage: 0.00%

Environment/s Reproduced in:

• Local: Windows 10 RS3 / Python 3.7.4
• Cloudbees Jenkins: Windows Server 2016 / Python 3.6

To Reproduce:

1. python -m venv venv
2. venv/scripts/activate
3. pip install -e “.[test]” --upgrade (see setup.py)
4. pytest tests/unit -v --cov mypkg --cov-report=html -r s -n 4 --cov-fail-under 80 --junit-xml=unit_test_results.xml
5. Observe failure.

To Work Around:

1. Remove --cov-fail-under 80 from command
2. Observe coverage report provides correct coverage. setup.py:

BUILD_NUM = os.environ.get('BUILD_NUMBER', '0')
VERSION = "0.2.0.{}".format(BUILD_NUM)

# setting pytest-cov to 2.7.1 to avoid issue
TEST_REQUIREMENTS = ['pytest', 'pytest-xdist', 'coverage', 'pytest-cov<=2.7.1', 'pylint-fail-under', 'pylint']

setup(
        name="mypkg",
        version=VERSION,
        description="description",
        packages=find_packages(
            exclude='tests/'
        ),
        tests_require=TEST_REQUIREMENTS,
        install_requires=[
            'requests',
            'requests_negotiate_sspi',
            'wincertstore',
            'click'
            ],
        extras_require={


            'test':TEST_REQUIREMENTS,
            'prototyping':'jupyter',
        })

pip list

Package                 Version   Location
----------------------- --------- ----------------------------
apipkg                  1.5
asn1crypto              0.24.0
astroid                 2.2.5
atomicwrites            1.3.0
attrs                   19.1.0
backcall                0.1.0
bcrypt                  3.1.7
bleach                  3.1.0
certifi                 2019.6.16
cffi                    1.12.3
chardet                 3.0.4
Click                   7.0
colorama                0.4.1
coverage                4.5.4
cryptography            2.7
decorator               4.4.0
defusedxml              0.6.0
entrypoints             0.3
execnet                 1.7.1
idna                    2.8
importlib-metadata      0.21
ipykernel               5.1.2
ipython                 7.8.0
ipython-genutils        0.2.0
ipywidgets              7.5.1
isort                   4.3.21
jedi                    0.15.1
Jinja2                  2.10.1
jsonschema              3.0.2
jupyter                 1.0.0
jupyter-client          5.3.1
jupyter-console         6.0.0
jupyter-core            4.5.0
lazy-object-proxy       1.4.2
MarkupSafe              1.1.1
mccabe                  0.6.1
mistune                 0.8.4
more-itertools          7.2.0
nbconvert               5.6.0
nbformat                4.4.0
notebook                6.0.1
packaging               19.1
pandocfilters           1.4.2
paramiko                2.6.0
parso                   0.5.1
pickleshare             0.7.5
pip                     19.0.3
pluggy                  0.13.0
prometheus-client       0.7.1
prompt-toolkit          2.0.9
py                      1.8.0
mypkg               0.2.0.0   d:\code\owr\devops\mypkg
pycparser               2.19
Pygments                2.4.2
pylint                  2.3.1
pylint-fail-under       0.2.0
PyNaCl                  1.3.0
pyparsing               2.4.2
pypiwin32               223
pyrsistent              0.15.4
pytest                  5.1.2
pytest-cov              2.8.0 # directly upgraded to repro locally
pytest-forked           1.0.2
pytest-xdist            1.29.0
python-dateutil         2.8.0
pywin32                 224
pywinpty                0.5.5
pyzmq                   18.1.0
qtconsole               4.5.5
requests                2.22.0
requests-negotiate-sspi 0.5.2
Send2Trash              1.5.0
setuptools              40.8.0
six                     1.12.0
terminado               0.8.2
testpath                0.4.2
tornado                 6.0.3
traitlets               4.3.2
typed-ast               1.4.0
urllib3                 1.25.3
wcwidth                 0.1.7
webencodings            0.5.1
widgetsnbextension      3.5.1
wincertstore            0.2
wrapt                   1.11.2
zipp                    0.6.0

[ionelmc]

Released 2.8.1. Off to get drunk in a dirty bar! 🎉

[Integralist]

@ionelmc hello 👋

Just to make reference to something you said…

So it looks that adding in --cov-report triggers the bug.

I’ve updated my deps (see above comment) and I still get the incorrect coverage reported.

I then additionally removed --cov-report so the default behaviour is applied and even then I still get the same FAIL Required test coverage of 85% not reached. Total coverage: 45.39%.

🤔