Cannot unify array(float64, 1d, C) and Literal[int](1)

Reporting a bug

• I have tried using the latest released version of Numba: 0.52.0.
• I have included a self contained code sample to reproduce the problem.

I have tried running some old code that was working fine about a year ago (on Numba 0.45 I believe), but obtained an error on Numba 0.52.0:

Traceback (most recent call last):
  File "test_numba.py", line 113, in <module>
    test_simplex_projection()
  File "test_numba.py", line 100, in test_simplex_projection
    x = utils.simplex_projection_inequality_sparse(A.data, A.indices, A.indptr, A.shape[0])
  File "/Users/user/.env/py37/lib/python3.7/site-packages/numba/core/dispatcher.py", line 414, in _compile_for_args
    error_rewrite(e, 'typing')
  File "/Users/user/.env/py37/lib/python3.7/site-packages/numba/core/dispatcher.py", line 357, in error_rewrite
    raise e.with_traceback(None)
numba.core.errors.TypingError: Failed in nopython mode pipeline (step: nopython frontend)
Cannot unify array(float64, 1d, C) and Literal[int](1) for 'xi.3', defined at /Users/user/github/netw0rkf10w/pyADGM/tests/../adgm/utils.py (172)

File "../adgm/utils.py", line 172:
def simplex_projection_inequality_sparse(data, indices, indptr, num_vectors):
    <source elided>
                    break
        x[start:end] = xi
        ^

During: typing of assignment at /Users/user/github/netw0rkf10w/pyADGM/tests/../adgm/utils.py (172)

File "../adgm/utils.py", line 172:
def simplex_projection_inequality_sparse(data, indices, indptr, num_vectors):
    <source elided>
                    break
        x[start:end] = xi
        ^

Below is the code for reproducing the issue. I am on macOS, Python 3.7.

# https://github.com/netw0rkf10w/pyADGM
import numpy as np
import numba
import scipy.sparse

@numba.jit(nopython=True, parallel=True, cache=True)
def simplex_projection_inequality_sparse(data, indices, indptr, num_vectors):
    """
    simplex project (with inequality constraints) of each row or each column of a sparse matrix C
    If C is CSR: each row; if C is CSC: each column
    data, indices, indptr, shape: representation of C (same notation as scipy csr/csc matrix)
    num_vectors = number of rows if C is CSR
                = number of cols if C is CSC
    """
    x = np.zeros(len(data))
    for i in numba.prange(num_vectors):
        # projection for each row independently
        start = indptr[i]
        end = indptr[i+1]
        if end <= start:
            continue
        ci = data[start:end]
        u = np.maximum(ci, 0)
        if np.sum(u) <= 1:
            xi = u
        else:
            ni = end - start
            a = -np.sort(-ci)
            lambdas = (np.cumsum(a) - 1)/np.arange(1, ni+1)
            xi = 1
            for k in range(ni-1, -1, -1):
                if a[k] > lambdas[k]:
                    xi = np.maximum(ci - lambdas[k], 0)
                    break
        x[start:end] = xi
    return x


def test_simplex_projection():
    A = scipy.sparse.random(4, 5, density=0.5, format='csr')
    x = simplex_projection_inequality_sparse(A.data, A.indices, A.indptr, A.shape[0])
    print(x)

if __name__ == "__main__":
    test_simplex_projection()

Thank you very much in advance!

[stuartarchibald]

Thanks for the report. Numba 0.49 moved to using full SSA form. This massively improved correctness and makes analysis etc a lot easier. The reason your code worked in prior versions of Numba is likely an accident due to the part-way-to-SSA form present in the algorithms Numba was using. You’ll note in your code that xi is sometimes a 1d array of floats (as returned by e.g. np.maximum) and on another occasion it’s the value 1. Essentially this code is not type stable and Numba is trying to tell you.

Adjusting the code to something like this permits compilation:

import numpy as np
import numba
import scipy.sparse

@numba.jit(nopython=True, parallel=True, cache=True)
def simplex_projection_inequality_sparse(data, indices, indptr, num_vectors):
    """
    simplex project (with inequality constraints) of each row or each column of a sparse matrix C
    If C is CSR: each row; if C is CSC: each column
    data, indices, indptr, shape: representation of C (same notation as scipy csr/csc matrix)
    num_vectors = number of rows if C is CSR
                = number of cols if C is CSC
    """
    x = np.zeros(len(data))
    for i in numba.prange(num_vectors):
        # projection for each row independently
        start = indptr[i]
        end = indptr[i+1]
        if end <= start:
            continue
        ci = data[start:end]
        u = np.maximum(ci, 0)
        if np.sum(u) <= 1:
            xi = u
        else:
            ni = end - start
            a = -np.sort(-ci)
            lambdas = (np.cumsum(a) - 1)/np.arange(1, ni+1)
            xi = np.ones(1,)  # <--------- CHANGE IS HERE
            for k in range(ni-1, -1, -1):
                if a[k] > lambdas[k]:
                    xi = np.maximum(ci - lambdas[k], 0)
                    break
        x[start:end] = xi
    return x


def test_simplex_projection():
    A = scipy.sparse.random(4, 5, density=0.5, format='csr')
    x = simplex_projection_inequality_sparse(A.data, A.indices, A.indptr, A.shape[0])
    print(x)

if __name__ == "__main__":
    test_simplex_projection()

Hope this helps?

[alisheikholeslam]

Hello @stuartarchibald I have a similar issue with the following code:

@nb.jit("int32[:, ::1], float64[:, ::1]")
def new(cv, vc):
   for i, j in enumerate(cv):
      coord = []
      for k in j:
         coord.append(vc[k])
      min_coord = np.amin(coord, axis=0)

   return min_coord

which will shows amin(list(array(float64, 1d, C))<iv=None>, axis=Literal[int](0)) Why it is happed and what is the solution? data.zip