asyncio.ProcessPoolExecutor tracing not working correctly

Originally reported by Alexander Mohr (Bitbucket: thehesiod, GitHub: thehesiod)

---

first I had to monkey patch concurrent.futures.process._process_worker from concurrent.futures.ProcessPoolExecutor because it doesn’t call the atexit handlers:

#!python

def _concurrent_futures_process_worker(*args, **kwargs):
    orig_call = kwargs.pop('_orig_call')
    result = orig_call(*args, **kwargs)

    for aeh in _concurrent_futures_process_worker._atexit_handlers:
        aeh()

    return result

_concurrent_futures_process_worker._atexit_handlers = []

def _init_coverage_monkey_patch():
    try:
        import coverage
        cps = os.environ.get("COVERAGE_PROCESS_START")
        if not cps:
            return

        # process pool executors don't call atexit handlers :(
        concurrent.futures.process._process_worker = functools.partial(_concurrent_futures_process_worker, _orig_call=concurrent.futures.process._process_worker)
    except:
        pass

def start_coverage():
    try:
       import coverage

        cps = os.environ.get("COVERAGE_PROCESS_START")
        if not cps:
            # No request for coverage, nothing to do.
            return

        cov = coverage.Coverage(config_file=cps, auto_data=True)
        cov.start()
        cov._warn_no_data = False
        cov._warn_unimported_source = False

        _concurrent_futures_process_worker._atexit_handlers.append(cov._atexit)
    except:
        pass

my first note is that this monkey patch would be a lot simpler if coverage.process_startup() returned the Coverage instance.

With the above patching, and running with “-p” for parallel, I actually get coverage data, however it’s all bizarre. For example: in a function it will say the first line executed, but the next line did not, and the loop.run_until_complete line did not execute, but most of the lines in the coroutine did.

---

• Bitbucket: https://bitbucket.org/ned/coveragepy/issue/481

[nedbat]

Thanks, I’m looking into this. BTW, I took your suggestion of returning the Coverage instance from process_startup. That’s in f3b4faef8280 (bb).

[nedbat]

Can you provide a reproducible description of the problem before we jump into the solution?