Numba is raising a `NotImplementedError` instead of the `NumbaNotImplementedError` equivalent

Reporting a bug

• I have tried using the latest released version of Numba (most recent is visible in the change log (https://github.com/numba/numba/blob/master/CHANGE_LOG).
• I have included a self contained code sample to reproduce the problem. i.e. it’s possible to run as ‘python bug.py’.

While trying the new error model with RBC, I found a NotImplementedError left on numba/core/base.py which I suspect it should be a errors.NumbaNotImplementedError. This causes the compilation process to crash because the exception is not captured anymore.

[stuartarchibald]

I’m unconvinced that there’s a noninvasive way to fix this whilst supporting the new error model behaviours.

I’m also unconvinced that there’s a way to fix the current logic that doesn’t involve turning the fall through NotImplementedError into a errors.NumbaNotImplementedError. The knock-on effect doing this is that there will be no “hard error” in the new error model and instead a standard LoweringError would be raised with this error captured inside it. This is not ideal but is probably acceptable as this is about finding a lowering implementation opposed to dealing with a problem in compiling one. Further the error information is not lost, it’s just wrapped, which was in part the aim of the new error handling behaviours (information preservation).

I think this problem in general stems from using exceptions for control flow purposes and the fall through raising of an exception being used as part of that control flow in a valid case (the one above). That there’s multiple recursion points also makes this harder to debug/understand.

The other option to fix this is to rewrite this logic to match what is needed for the current Numba code base (IIRC this function has not been updated for a long while, whilst most of the code that uses it has changed drastically). This may well be a better option in the long term albeit potentially a lot of effort.

[stuartarchibald]

Applying this patch to mainline:

diff --git a/numba/core/base.py b/numba/core/base.py
index e3b55c0..7891ca6 100644
--- a/numba/core/base.py
+++ b/numba/core/base.py
@@ -539,34 +539,47 @@ class BaseContext(object):
         Return the implementation of function *fn* for signature *sig*.
         The return value is a callable with the signature (builder, args).
         """
+        def printit(where, firstcall, func):
+            fmt = '{:<25} {:<5} {:<40} {:<40}'
+            print(fmt.format(where, str(firstcall), str(func), str(type(func))))
         assert sig is not None
         sig = sig.as_function()
         if isinstance(fn, (types.Function, types.BoundFunction,
                            types.Dispatcher)):
+            printit("1 Branch", _firstcall, fn)
             key = fn.get_impl_key(sig)
             overloads = self._defns[key]
         else:
+            printit("2 Branch", _firstcall, fn)
             key = fn
             overloads = self._defns[key]
 
         try:
+            printit("3 Try", _firstcall, fn)
             return _wrap_impl(overloads.find(sig.args), self, sig)
         except errors.NumbaNotImplementedError:
+            printit("4 Except", _firstcall, fn)
             pass
         if isinstance(fn, types.Type):
+            printit("5 Branch", _firstcall, fn)
             # It's a type instance => try to find a definition for the type class
             try:
+                printit("6 Try recurse", _firstcall, fn)
                 return self.get_function(type(fn), sig)
             except errors.NumbaNotImplementedError:
+                printit("7 Except", _firstcall, fn)
                 # Raise exception for the type instance, for a better error message
                 pass
 
+        printit("8 Fall through", _firstcall, fn)
         # Automatically refresh the context to load new registries if we are
         # calling the first time.
         if _firstcall:
+            printit("9 Branch refresh recurse", _firstcall, fn)
             self.refresh()
             return self.get_function(fn, sig, _firstcall=False)
 
+        printit("10 Fall through raise", _firstcall, fn)
         raise NotImplementedError("No definition for lowering %s%s" % (key, sig))

gives:

$ NUMBA_CAPTURED_ERRORS='old_style' python issue7507.py 
1 Branch                  True  Function(<function lround at 0x7f83e61be310>) <class 'numba.core.types.functions.Function'>
3 Try                     True  Function(<function lround at 0x7f83e61be310>) <class 'numba.core.types.functions.Function'>
4 Except                  True  Function(<function lround at 0x7f83e61be310>) <class 'numba.core.types.functions.Function'>
5 Branch                  True  Function(<function lround at 0x7f83e61be310>) <class 'numba.core.types.functions.Function'>
6 Try recurse             True  Function(<function lround at 0x7f83e61be310>) <class 'numba.core.types.functions.Function'>
2 Branch                  True  <class 'numba.core.types.functions.Function'> <class 'numba.core.types.abstract._TypeMetaclass'>
3 Try                     True  <class 'numba.core.types.functions.Function'> <class 'numba.core.types.abstract._TypeMetaclass'>
4 Except                  True  <class 'numba.core.types.functions.Function'> <class 'numba.core.types.abstract._TypeMetaclass'>
8 Fall through            True  <class 'numba.core.types.functions.Function'> <class 'numba.core.types.abstract._TypeMetaclass'>
9 Branch refresh recurse  True  <class 'numba.core.types.functions.Function'> <class 'numba.core.types.abstract._TypeMetaclass'>
2 Branch                  False <class 'numba.core.types.functions.Function'> <class 'numba.core.types.abstract._TypeMetaclass'>
3 Try                     False <class 'numba.core.types.functions.Function'> <class 'numba.core.types.abstract._TypeMetaclass'>
4 Except                  False <class 'numba.core.types.functions.Function'> <class 'numba.core.types.abstract._TypeMetaclass'>
8 Fall through            False <class 'numba.core.types.functions.Function'> <class 'numba.core.types.abstract._TypeMetaclass'>
10 Fall through raise     False <class 'numba.core.types.functions.Function'> <class 'numba.core.types.abstract._TypeMetaclass'>
...
numba/core/base.py", line 583, in get_function
    raise NotImplementedError("No definition for lowering %s%s" % (key, sig))
numba.core.errors.LoweringError: Failed in nopython mode pipeline (step: native lowering)
No definition for lowering <class 'numba.core.types.functions.Function'>(float64,) -> int64

File "issue7507.py", line 27:
def foo(a):
    return lround(a)
    ^

and:

$ NUMBA_CAPTURED_ERRORS='new_style' python issue7507.py 
1 Branch                  True  Function(<function lround at 0x7fe45ff00310>) <class 'numba.core.types.functions.Function'>
3 Try                     True  Function(<function lround at 0x7fe45ff00310>) <class 'numba.core.types.functions.Function'>
4 Except                  True  Function(<function lround at 0x7fe45ff00310>) <class 'numba.core.types.functions.Function'>
5 Branch                  True  Function(<function lround at 0x7fe45ff00310>) <class 'numba.core.types.functions.Function'>
6 Try recurse             True  Function(<function lround at 0x7fe45ff00310>) <class 'numba.core.types.functions.Function'>
2 Branch                  True  <class 'numba.core.types.functions.Function'> <class 'numba.core.types.abstract._TypeMetaclass'>
3 Try                     True  <class 'numba.core.types.functions.Function'> <class 'numba.core.types.abstract._TypeMetaclass'>
4 Except                  True  <class 'numba.core.types.functions.Function'> <class 'numba.core.types.abstract._TypeMetaclass'>
8 Fall through            True  <class 'numba.core.types.functions.Function'> <class 'numba.core.types.abstract._TypeMetaclass'>
9 Branch refresh recurse  True  <class 'numba.core.types.functions.Function'> <class 'numba.core.types.abstract._TypeMetaclass'>
2 Branch                  False <class 'numba.core.types.functions.Function'> <class 'numba.core.types.abstract._TypeMetaclass'>
3 Try                     False <class 'numba.core.types.functions.Function'> <class 'numba.core.types.abstract._TypeMetaclass'>
4 Except                  False <class 'numba.core.types.functions.Function'> <class 'numba.core.types.abstract._TypeMetaclass'>
8 Fall through            False <class 'numba.core.types.functions.Function'> <class 'numba.core.types.abstract._TypeMetaclass'>
10 Fall through raise     False <class 'numba.core.types.functions.Function'> <class 'numba.core.types.abstract._TypeMetaclass'>
...
numba/core/base.py", line 583, in get_function
    raise NotImplementedError("No definition for lowering %s%s" % (key, sig))
NotImplementedError: No definition for lowering <class 'numba.core.types.functions.Function'>(float64,) -> int64

which I think confirms this issue is invariant of the error handler.