Improve doc/subprocess.rst for multiprocessing module limitation
See original GitHub issueIs your feature request related to a problem? Please describe.
I am developing a program which leverage multiprocessing
module to do parallel tasks. After read doc/subprocess.rst
and try, I figured out that documented solution doesn’t work for multiprocessing.Process
.Fisrt, both sitecustomize.py
and .pth
file doesn’t work in subprocesses created by multiprocessing.Process
. It’s because when use multiprocessing.Process
, subprocesses won’t import site
module again, there is not chance to do sitecustomize.py
or .pth
file customization. Second, when these subprocesses exit, atexit
functions get never called, coverage
has no chance to write collected data.
Describe the solution you’d like Maybe we could improve document and point out this limitation.
Describe alternatives you’ve considered I came up with a workaround.
test.py
def func():
try:
import coverage
coverage.process_startup()
except:
traceback.print_exc()
# Emulate time-consuming work
time.sleep(5)
atexit._run_exitfuncs()
p = Process(target=func)
p.start()
p.join()
COVERAGE_PROCESS_START=.coveragerc coverage run test.py
This seems work. Apparently, there are many cons. 1. there may exists side-effects when other libraries depends on this atexit feature. 2. We call atexit protected method. 3. At most times, we can not (or want to avoid to) add such code into normal code.
Additional context None
Issue Analytics
- State:
- Created 4 years ago
- Comments:6 (4 by maintainers)
Done: 5379317ad73694099b7876797695bc76bd9cdce5
I should add more mention of it to the docs.