question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Feature request: Disable Numba parallelism

See original GitHub issue

Since I’m interested in comparing multiple UMAP results, I tried to run one UMAP instance per thread - which led to all of them being deadlocked. After some digging I found that changing the Numba decorators in umap_.py from @numba.njit(parallel=True, fastmath=True) to @numba.njit(parallel=False, fastmath=True), i. e. disabling the Numba parallelism, resolved the deadlock. Perhaps this is related to some open Numba thread coordination/deadlock issues (see e. g. https://github.com/numba/numba/issues/2804).

A configuration option to disable Numba parallelism would be nice, since it enables the user to decide whether he/she wants to use Numba to parallelize or do it manually (e. g. running one UMAP model per thread) or not at all.

Issue Analytics

  • State:open
  • Created 5 years ago
  • Reactions:3
  • Comments:24 (9 by maintainers)

github_iconTop GitHub Comments

2reactions
stuartarchibaldcommented, Sep 18, 2018

@lmcinnes Numba 0.40.0 rc1 was published last week if you want to give it a try https://groups.google.com/a/continuum.io/forum/#!topic/numba-users/pYAd-kT1mDM? The official release will be published shortly.

2reactions
stuartarchibaldcommented, Aug 21, 2018

https://github.com/numba/numba/pull/3202 is WIP to address threadsafety in the Numba parallel backend.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Automatic parallelization with @jit
Setting the parallel option for jit() enables a Numba transformation pass that attempts to automatically parallelize and perform other optimizations on (part of) ......
Read more >
Troubleshooting and tips — Numba 0.50.1 documentation
In order to debug code, it is possible to disable JIT compilation, which makes the jit decorator (and the njit decorator) act as...
Read more >
Supported Python features
Numba strives to support as much of the Python language as possible, but some language features are not available inside Numba-compiled functions.
Read more >
Environment variables
Disable JIT compilation entirely. The jit() decorator acts as if it performs no operation, and the invocation of decorated functions calls the original...
Read more >
The Threading Layers — Numba 0.50.1 documentation
The use of the parallel=True kwarg in @jit and @njit . ... portability concerns, the OpenMP threading layer is disabled in the Numba...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found