UMAP broken with numba==0.43

Hi @lmcinnes, I wanted to let you know about a new dependency issue that just popped up. It looks like the most recent Numba release and UMAP don’t play nice together. There was a ton of updates included in the Numba release notes, so it’s not clear what exactly the problem is.

With Python 3.7.1, umap-learn==0.3.7, and numba==0.43.0 (released March 13th), this simple UMAP test script produces the following error:

import umap
import numpy as np
r = np.random.random((100,3))
r2d = umap.UMAP().fit_transform(r)

Traceback (most recent call last):
  File "test.py", line 4, in <module>
    r2d = umap.UMAP().fit_transform(r)
  File "/Users/sauln/research/scratch/test_umap/venv/lib/python3.7/site-packages/umap/umap_.py", line 1566, in fit_transform
    self.fit(X, y)
  File "/Users/sauln/research/scratch/test_umap/venv/lib/python3.7/site-packages/umap/umap_.py", line 1398, in fit
    self.verbose,
  File "/Users/sauln/research/scratch/test_umap/venv/lib/python3.7/site-packages/numba/dispatcher.py", line 350, in _compile_for_args
    error_rewrite(e, 'typing')
  File "/Users/sauln/research/scratch/test_umap/venv/lib/python3.7/site-packages/numba/dispatcher.py", line 317, in error_rewrite
    reraise(type(e), e, None)
  File "/Users/sauln/research/scratch/test_umap/venv/lib/python3.7/site-packages/numba/six.py", line 658, in reraise
    raise value.with_traceback(tb)
numba.errors.TypingError: Failed in nopython mode pipeline (step: nopython frontend)
Unknown attribute 'shape' of type none

File "venv/lib/python3.7/site-packages/umap/umap_.py", line 88:
def smooth_knn_dist(distances, k, n_iter=64, local_connectivity=1.0, bandwidth=1.0):
    <source elided>
    target = np.log2(k) * bandwidth
    rho = np.zeros(distances.shape[0])
    ^

[1] During: typing of get attribute at /Users/sauln/research/scratch/test_umap/venv/lib/python3.7/site-packages/umap/umap_.py (88)

File "venv/lib/python3.7/site-packages/umap/umap_.py", line 88:
def smooth_knn_dist(distances, k, n_iter=64, local_connectivity=1.0, bandwidth=1.0):
    <source elided>
    target = np.log2(k) * bandwidth
    rho = np.zeros(distances.shape[0])
    ^

This is not usually a problem with Numba itself but instead often caused by
the use of unsupported features or an issue in resolving types.

To see Python/NumPy features supported by the latest release of Numba visit:
http://numba.pydata.org/numba-doc/dev/reference/pysupported.html
and
http://numba.pydata.org/numba-doc/dev/reference/numpysupported.html

For more information about typing errors and how to debug them visit:
http://numba.pydata.org/numba-doc/latest/user/troubleshoot.html#my-code-doesn-t-compile

If you think your code should work with Numba, please report the error message
and traceback, along with a minimal reproducer at:
https://github.com/numba/numba/issues/new

After downgrading to numba==0.42, everything ran perfectly fine, so this is a fine temporary solution.

[kramred]

It’s a bug in Numba 0.43: https://github.com/numba/numba/issues/3879

You can work around the bug by introducing the following change in umap_.py (although you probably shouldn’t):

diff --git a/umap/umap_.py b/umap/umap_.py
index b37469a..f488dec 100644
--- a/umap/umap_.py
+++ b/umap/umap_.py
@@ -445 +445 @@ def fuzzy_simplicial_set(
-    if knn_indices is None or knn_dists is None:
+    if (not knn_indices) or (not knn_dists):

[stuartarchibald]

Fix to the invalid mutation in the fall back compilation path is here: https://github.com/numba/numba/pull/3883