Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

No parallelization during nearest neighbors computation

See original GitHub issue

I’m using scanpy with the following version

scanpy 1.4.4.post1
numba 0.45.1
numpy 1.17.2

on a Ubuntu virtual machine with 16 cpus. In the example data below, I have 47933 cells × 41 genes. When I run

sc.pp.neighbors(adata, n_neighbors=100, n_pcs=12)

it empirically uses only 1 core and takes 1 min 35s and I get the warning:

/opt/miniconda3/envs/py37_2/lib/python3.7/site-packages/numba/compiler.py:602: NumbaPerformanceWarning: 
The keyword argument 'parallel=True' was specified but no transformation for parallel execution was possible.

To find out why, try turning on parallel diagnostics, see http://numba.pydata.org/numba-doc/latest/user/parallel.html#diagnostics for help.

File "../../../../../opt/miniconda3/envs/py37_2/lib/python3.7/site-packages/umap/rp_tree.py", line 135:
@numba.njit(fastmath=True, nogil=True, parallel=True)
def euclidean_random_projection_split(data, indices, rng_state):
^

  self.func_ir.loc))
/opt/miniconda3/envs/py37_2/lib/python3.7/site-packages/umap/nndescent.py:92: NumbaPerformanceWarning: 
The keyword argument 'parallel=True' was specified but no transformation for parallel execution was possible.

To find out why, try turning on parallel diagnostics, see http://numba.pydata.org/numba-doc/latest/user/parallel.html#diagnostics for help.

File "../../../../../opt/miniconda3/envs/py37_2/lib/python3.7/site-packages/umap/utils.py", line 409:
@numba.njit(parallel=True)
def build_candidates(current_graph, n_vertices, n_neighbors, max_candidates, rng_state):
^

  current_graph, n_vertices, n_neighbors, max_candidates, rng_state
/opt/miniconda3/envs/py37_2/lib/python3.7/site-packages/numba/compiler.py:602: NumbaPerformanceWarning: 
The keyword argument 'parallel=True' was specified but no transformation for parallel execution was possible.

To find out why, try turning on parallel diagnostics, see http://numba.pydata.org/numba-doc/latest/user/parallel.html#diagnostics for help.

File "../../../../../opt/miniconda3/envs/py37_2/lib/python3.7/site-packages/umap/nndescent.py", line 47:
    @numba.njit(parallel=True)
    def nn_descent(
    ^

  self.func_ir.loc))

when I run

from joblib import parallel_backend
with parallel_backend('threading', n_jobs=15):
    sc.pp.neighbors(adata, n_neighbors=100, n_pcs=12)

as suggested in #659, it takes 1 min 28 seconds, empirically mostly uses 1 cpu, and gives the following warning

/opt/miniconda3/envs/py37_2/lib/python3.7/site-packages/numba/compiler.py:602: NumbaPerformanceWarning: 
The keyword argument 'parallel=True' was specified but no transformation for parallel execution was possible.

To find out why, try turning on parallel diagnostics, see http://numba.pydata.org/numba-doc/latest/user/parallel.html#diagnostics for help.

File "../../../../../opt/miniconda3/envs/py37_2/lib/python3.7/site-packages/umap/rp_tree.py", line 135:
@numba.njit(fastmath=True, nogil=True, parallel=True)
def euclidean_random_projection_split(data, indices, rng_state):
^

  self.func_ir.loc))
/opt/miniconda3/envs/py37_2/lib/python3.7/site-packages/numba/compiler.py:602: NumbaPerformanceWarning: 
The keyword argument 'parallel=True' was specified but no transformation for parallel execution was possible.

To find out why, try turning on parallel diagnostics, see http://numba.pydata.org/numba-doc/latest/user/parallel.html#diagnostics for help.

File "../../../../../opt/miniconda3/envs/py37_2/lib/python3.7/site-packages/umap/nndescent.py", line 47:
    @numba.njit(parallel=True)
    def nn_descent(
    ^

  self.func_ir.loc))

Are there any tips on how I can benefit from parallelization in these nearest neighbor computations? This is the bottleneck in my work flow.

Issue Analytics

State:
Created 4 years ago
Comments:16 (6 by maintainers)

Top GitHub Comments

1reaction

tomwhitecommented, Nov 13, 2019

Ah yes - sorry I missed the context manager bit from earlier in this thread. I tried running this earlier today using the script in theislab/scanpy_usage#17, and for 130K cells NN took 53s unoptimized vs 35s optimized (32 cores). (Not a proportionate speedup, but still worthwhile.) The UMAP speedup shown in that script is significant too.

I see that there is a UMAP issue to discuss this further.

0reactions

dylkotcommented, Nov 13, 2019

@tomwhite, thanks for posting this script. The setup steps ultimately seem to work for me and to produce code that uses all of the allotted cores but it also increases the memory overhead dramatically such that I need >250 GB of RAM which doesn’t seem worth it in this case.

Top Results From Across the Web

No parallelization during nearest neighbors computation #913

I'm using scanpy with the following version scanpy 1.4.4.post1 numba 0.45.1 numpy 1.17.2 on a Ubuntu virtual machine with 16 cpus. In the ......

parallel algorithms for nearest neighbor search problems in ...

Direct nearest neighbors. We propose two parallel algorithms for the direct calculation of the KNN problem. The first one prioritizes computing time over...

Parallel k-Nearest Neighbor - Ali Tarhini - WordPress.com

However, in this paper the KNN algorithm was parallelized on CPU by distributing the distance computations of the k nearest neighbors among ...

Applying Parallel Processing to Improve the Computation ...

Abstract: K-Nearest Neighbor (KNN) is a widely used algorithm to gain an accurate and efficient classification. One of the drawbacks of the algorithm...

Parallel implementations of the False Nearest Neighbors ...

A key point in this process is the computing of the time series embedding dimension using the False Nearest Neighbors (FNN) method. The...