Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Multiprocessing integration

See original GitHub issue

We do support fork() in general, but the multiprocessing module has some quirks.

atexit does not get called, so events get lost
Unhandled exceptions are not captured

An integration should fix both. We should also consider Billard, a fork of mulitprocessing here. If we manage to properly hook into both versions, we can catch some startup errors in Celery as well.

If you’re interested in this, please give this post a 👍 so it’s easier to track.

Issue Analytics

State:
Created 5 years ago
Reactions:38
Comments:6 (3 by maintainers)

Top GitHub Comments

1reaction

untitakercommented, Mar 20, 2019

@crepererum could you open a new issue wrt Dask support? The more I think about this (multiprocessing) support the more I thikn it’s an A/B problem.

0reactions

github-actions[bot]commented, Dec 23, 2021

This issue has gone three weeks without activity. In another week, I will close it.

But! If you comment or otherwise update it, I will reset the clock, and if you label it Status: Backlog or Status: In Progress, I will leave it alone … forever!

“A weed is but an unloved flower.” ― Ella Wheeler Wilcox 🥀

Top Results From Across the Web

multiprocessing — Process-based parallelism — Python 3.11 ...

multiprocessing is a package that supports spawning processes using an API similar to the threading module. The multiprocessing package offers both local ...

Python multiprocessing - process-based parallelism in Python

Python multiprocessing tutorial is an introductory tutorial to process-based parallelism in ... Monte Carlo analysis; numerical integration ...

Multiprocessing - Python Numerical Methods

The multiprocessing library is the Python's standard library to support parallel computing using processes. It has many different features, if you want to ......

Multiprocessing in Python | Data Integration

Multiprocessing is the ability of a system to run multiple processors at one time. If you had a computer with a single processor, ......

How to implement multiprocessing in Monte Carlo integration

2. You basically need to split your range and then use something like pool.map to solve the subranges independently. · 1. You should...