Feature request: Disable Numba parallelism
See original GitHub issueSince I’m interested in comparing multiple UMAP results, I tried to run one UMAP instance per thread - which led to all of them being deadlocked. After some digging I found that changing the Numba decorators in umap_.py
from @numba.njit(parallel=True, fastmath=True)
to @numba.njit(parallel=False, fastmath=True)
, i. e. disabling the Numba parallelism, resolved the deadlock. Perhaps this is related to some open Numba thread coordination/deadlock issues (see e. g. https://github.com/numba/numba/issues/2804).
A configuration option to disable Numba parallelism would be nice, since it enables the user to decide whether he/she wants to use Numba to parallelize or do it manually (e. g. running one UMAP model per thread) or not at all.
Issue Analytics
- State:
- Created 5 years ago
- Reactions:3
- Comments:24 (9 by maintainers)
Top Results From Across the Web
Automatic parallelization with @jit
Setting the parallel option for jit() enables a Numba transformation pass that attempts to automatically parallelize and perform other optimizations on (part of) ......
Read more >Troubleshooting and tips — Numba 0.50.1 documentation
In order to debug code, it is possible to disable JIT compilation, which makes the jit decorator (and the njit decorator) act as...
Read more >Supported Python features
Numba strives to support as much of the Python language as possible, but some language features are not available inside Numba-compiled functions.
Read more >Environment variables
Disable JIT compilation entirely. The jit() decorator acts as if it performs no operation, and the invocation of decorated functions calls the original...
Read more >The Threading Layers — Numba 0.50.1 documentation
The use of the parallel=True kwarg in @jit and @njit . ... portability concerns, the OpenMP threading layer is disabled in the Numba...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
@lmcinnes Numba 0.40.0 rc1 was published last week if you want to give it a try https://groups.google.com/a/continuum.io/forum/#!topic/numba-users/pYAd-kT1mDM? The official release will be published shortly.
https://github.com/numba/numba/pull/3202 is WIP to address threadsafety in the Numba parallel backend.