pytest and multiprocessing: CoverageException: Can't combine line data with arc dat

Originally reported by Samuel Colvin (Bitbucket: samuelcolvin, GitHub: samuelcolvin)

---

I’m getting CoverageException: Can't combine line data with arc dat when using coverage with the pytest-cov pytest extension. I think this is similar to #399, but that issue is closed.

Example failure on travis: https://travis-ci.org/samuelcolvin/arq/builds/148541711

The test causing problems is using click’s CliRunner, code here. The CLI command that’s testing starts another process (using multiprocessing.Process) to run the worker which is what’s causing the problem, if you prevent the worker process being started the exception doesn’t occur.

I tried changing the concurrency mode but it made no difference.

To reproduce: clone, pip install -e . && pip install -r tests/requirements.txt, py.test --cov=arq.

relevant bits of pip freeze:

click==6.6
coverage==4.2
pytest==2.9.2
pytest-cov==2.3.0

Let me know if you need anymore information.

---

• Bitbucket: https://bitbucket.org/ned/coveragepy/issue/512

[nedbat]

Original comment by Ionel Cristian Mărieș (Bitbucket: ionelmc, GitHub: ionelmc)

---

I’ve taken a look at this as well, my opinion is that a combination of configuration, subprocess use and cwd changes cause the problem. That’s why you get the data file saved without arcs (branch=False).

There are thee ways to deal with this:

• Specify --cov-config=setup.cfg (pytest-cov will absolutize it before tmpworkdir is used). Alternatively, you could use a .coveragerc (pytest-cov absolutize that if it exists).
• Fix the test to not use subprocesses. Afaik the test spawns a suprocess that sigterms the parent. This makes no sense to me - you can do that with no subprocess at all. Eg: signal.kill(os.getpid(), signal.SIGTERM) or even a thread.
• Stop changing current working directory (tmpworkdir fixture).

There is no bug in either coverage or pytest-cov here.

Also, note that concurrency = multiprocessing is completely un-necessary if you use pytest-cov (it completely manages coverage measurements in subprocesses for you).

[wohali]

I get this error when I use branch = true. @wohali @joshfriend can you confirm this with your repo?

Yes, I use branch = true. As I understand it this changes the format of the file, as I mentioned above, so the results cannot be combined with runs that don’t include that. Something to do with multiprocess runs and the toml file not being included caused the mixed format issue for me.