automating the subprocess measurement setup

Originally reported by Tibor (Bitbucket: tibor_arpas, GitHub: Unknown)

---

Subprocesses in test suits are a common occurence. coverage.py has an execellent way to measure their coverage but the set-up instructions are quite scary: http://nedbatchelder.com/code/coverage/subprocess.html

Also in the time of tox, continuous integration, virtualizations and containerizations, the manual process is quite beside the point.

What could we do to completely automate the described process?

There is an article by Ned http://nedbatchelder.com/blog/201001/running_code_at_python_startup.html

and a suggestion from @geoffbache

#!python
# sitecustomize.py
import os, sys
currDir = os.path.dirname(__file__)
sys.path.remove(currDir)
del sys.modules["sitecustomize"]
try:
    import sitecustomize
finally:
    sys.path.insert(0, currDir)

That script fails on my machine (MacOS, Python 2.7) with

#!stacktrace

Traceback (most recent call last):
  File "/Users/tibor/.virtualenvs/tmon/bin/../lib/python2.7/site.py", line 703, in <module>
    main()
  File "/Users/tibor/.virtualenvs/tmon/bin/../lib/python2.7/site.py", line 694, in main
    execsitecustomize()
  File "/Users/tibor/.virtualenvs/tmon/bin/../lib/python2.7/site.py", line 548, in execsitecustomize
    import sitecustomize
  File "/Users/tibor/tmonworkspace/testmon/sitecustomize.py", line 8, in <module>
    sys.path.insert(0, currDir)
AttributeError: 'NoneType' object has no attribute 'path'

But I would hope that it’s the best approach.

---

• Bitbucket: https://bitbucket.org/ned/coveragepy/issue/367

[wsanchez]

@glyph I think you can achieve your coverage groups by setting COVERAGE_FILE to .../coverage.groupname and user parallel mode.

What I just did in Klein’s Tox config is to set COVERAGE_FILE={toxworkdir}/coverage.{envname}, which (with parallel mode turned on) generate will generate a bunch of {toxworkdir}/coverage.{envname}.XXXX files, that I combine after the test run. I can then set COVERAGE_FILE={toxworkdir}/coverage later to get a combined coverage file for all test runs.

[nedbat]

Original comment by Buck Evan (Bitbucket: bukzor, GitHub: bukzor)

---

The cov-core solution doesn’t uninstall whatsoever. In fact it causes the whole virtualenv to explode (ImportError) if you uninstall it.

The coverage-pth solution is better, but doesn’t handle setup.py develop aka pip install -e.

The solution found in python-hunter handles all those cases, in addition to supporting easy-install. https://github.com/ionelmc/python-hunter/blob/master/setup.py

We’ve recently adapted this into a package that fills the same problem space as coverage-pth, but is more reliable.

IMO this should be a default feature of coveragepy. While it will add one if-statement to all subsequent python startup time, that seems like a near-zero cost to me, and the additional use-cases supported are sizeable.