Coverage for installed packages + smarter reports

@schlamar I don’t know if this could be of any use to anyone and I didn’t open a pull request since it’d require small changes to both pytest-cov and cov-core repos. I found myself hacking around the coverage reporting behaviour manually all the time (like changing current folder to location of the package to avoid ugly paths) so I wondered if it could be done upstream.

If you run coverage on an installed package (e.g. via pytest --pyargs), or more generally on a package that’s not in your current folder, your coverage report would look super ugly like so:

Name                                                                 Stmts   Miss  Cover   Missing
------------------------------------------------------
/home/smirnov/anaconda/lib/python2.7/site-packages/arrow/__init__        4      0   100%
/home/smirnov/anaconda/lib/python2.7/site-packages/arrow/api            13      0   100%
/home/smirnov/anaconda/lib/python2.7/site-packages/arrow/arrow         295      0   100%
/home/smirnov/anaconda/lib/python2.7/site-packages/arrow/factory        58      0   100%
/home/smirnov/anaconda/lib/python2.7/site-packages/arrow/formatter      78      0   100%
/home/smirnov/anaconda/lib/python2.7/site-packages/arrow/locales       371      2    99%   1182, 1189
/home/smirnov/anaconda/lib/python2.7/site-packages/arrow/parser        172      0   100%
/home/smirnov/anaconda/lib/python2.7/site-packages/arrow/util            7      0   100%
------------------------------------------------------
TOTAL                      998      2    99%

The same will go to html/xml reports.

Wouldn’t it would be nice to have reports like this instead?

Name        Stmts   Miss  Cover   Missing
--------------------------------------------
__init__        4      0   100%
api            13      0   100%
arrow         295      0   100%
factory        58      0   100%
formatter      78      0   100%
locales       371      2    99%   1182, 1189
parser        172      0   100%
util            7      0   100%
--------------------------------------------
TOTAL                      998      2    99%

Would you consider adding something like a --compact-report option to pytest-cov which would then pass some flag to CovController.summary() so it can do something like this before generating reports:

prefix = os.path.commonprefix(cov.data.measured_files())
if prefix:
    cov.file_locator.relative_dir = prefix  # +9000 nice points for all reports

That’s pretty much the entire implementation 😃

I could open pull requests to cov-core and pytest-cov if you’d want.

Thanks.

[aldanor]

After all this time, I still couldn’t find a proper solution to this 😿

Running tests against a site-packages version of the package as opposed to the working source tree has many benefits, e.g. it may reveal problems in packaging. However it all ends up with ugly coverage and it’s unobvious how to fix it (unless I’m missing something).

[ionelmc]

@aldanor you got two options:

• Develop installs. But this means you need to run tests two times, with and without coverage (where you don’t do develop install).
• Use coverage’s report cli. Basically you run coverage report to get a cleaner report. See https://hynek.me/articles/testing-packaging/