Cannot use numpy.typing.NDArray type hints

Numpy v1.21.0 introduced runtime subscriptable aliases for ndarrays. The feature is really cool and I really like the direction that it seems to be headed in.

However, these new annotations do not seem compatible with the @beartype decorator.

System info

OS: Windows 10, 20H2 Python: 3.8.10 Beartype: 0.71.0 Numpy: 1.21.0

Sample Code

Running this sample code:

from beartype import beartype
import numpy as np
import numpy.typing as npt

@beartype
def foo(arr: npt.NDArray[np.float64]) -> npt.NDArray[np.float64]:
    return arr

foo(np.zeros(3))

Yields this error:

Traceback (most recent call last):
  File "C:\Users\Antyos\Code\Project\.venv\lib\site-packages\beartype\_util\cls\utilclstest.py", line 174, in die_unless_type_isinstanceable
    isinstance(None, cls)  # type: ignore[arg-type]
  File "C:\Users\Antyos\Code\Project\.venv\lib\site-packages\numpy\typing\_generic_alias.py", line 137, in __instancecheck__
    raise TypeError("isinstance() argument 2 cannot be a "
TypeError: isinstance() argument 2 cannot be a parameterized generic

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "C:\Users\Antyos\Code\Project\.venv\lib\site-packages\beartype\_decor\main.py", line 268, in beartype
    func_wrapper_code = generate_code(func_data)
  File "C:\Users\Antyos\Code\Project\.venv\lib\site-packages\beartype\_decor\_code\codemain.py", line 162, in generate_code
    code_params = _code_check_params(data)
  File "C:\Users\Antyos\Code\Project\.venv\lib\site-packages\beartype\_decor\_code\codemain.py", line 324, in _code_check_params
    hint = coerce_hint_pep(
  File "C:\Users\Antyos\Code\Project\.venv\lib\site-packages\beartype\_decor\_cache\cachehint.py", line 254, in coerce_hint_pep
    die_unless_hint(hint=hint, hint_label=hint_label)
  File "C:\Users\Antyos\Code\Project\.venv\lib\site-packages\beartype\_util\hint\utilhinttest.py", line 104, in die_unless_hint
    die_unless_hint_nonpep(hint=hint, hint_label=hint_label)
  File "C:\Users\Antyos\Code\Project\.venv\lib\site-packages\beartype\_util\hint\nonpep\utilhintnonpeptest.py", line 111, in die_unless_hint_nonpep
    die_unless_hint_nonpep_type(
  File "C:\Users\Antyos\Code\Project\.venv\lib\site-packages\beartype\_util\hint\nonpep\utilhintnonpeptest.py", line 200, in die_unless_hint_nonpep_type
    die_unless_type_isinstanceable(
  File "C:\Users\Antyos\Code\Project\.venv\lib\site-packages\beartype\_util\cls\utilclstest.py", line 208, in die_unless_type_isinstanceable
    raise exception_cls(exception_message) from exception
beartype.roar.BeartypeDecorHintNonPepException: @beartyped foo() parameter "arr" type hint numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]] uncheckable at runtime (i.e., not passable as 
second parameter to isinstance() due to raising "isinstance() argument 2 cannot be a parameterized generic" from metaclass __instancecheck__() method).

Knowing that Beartype leverages the Python built-in Annotated type, it roughly makes sense why this error would be occurring. While creating a Beartype numpy array using Annotated and lambda still has its uses, static type checkers such as Pylance/Pyright are now equipped to recognize this new Numpy typing syntax (that’s how I initially found out about this change; my code had a ton of new errors I didn’t understand), but I don’t know if they can recognize the Beartype version in the same way.

[leycec]

awwww 🤗

Why are you so incredibly nice, that’s what I want to know! Relatedly, I actually thought of an even awfuller cleverer one-liner for portably detecting numpy.typing.NDArray type hints across Python versions: repr(hint).startswith('numpy.ndarray['). Despite being hideously obfuscatory and basically ineffable, it actually works:

$ ipython3.8
>>> import numpy as np
>>> import numpy.typing as npt
>>> hint = npt.NDArray[np.float64]
>>> repr(hint).startswith('numpy.ndarray[')
True

$ ipython3.9
>>> import numpy as np
>>> import numpy.typing as npt
>>> hint = npt.NDArray[np.float64]
>>> repr(hint).startswith('numpy.ndarray[')
True

I horrify even myself. 🙀

[leycec]

It is so nice to see the impressive pace at which beartype is developing!

aww, u so nice! 😊

Now that NDArray is supported by the latest release, one could add the support of ArrayLike to the roadmap.

Yes. Let’s add ever more back-breaking features onto my drooping shoulders. Actually, I think that’s just my bursitis acting up. Curse this geriatric body. 👴

In all seriousness, this is a great idea. So it shall be! I was hoping the remainder of the numpy.typing subpackage would work out-of-the-box. I am a utopian dreamer.

Lucky for me, the traceback is suggestive of an upstream issue. numpy.typing.ArrayLike appears to be inducing infinite recursion from within the typing module at runtime under at least Python < 3.9.0. This is reaaal sus af:

TypeError: typing.Union[typing.Sequence[typing.Sequence[typing.Sequence[
typing.Sequence[typing.Sequence[typing.Any]]]]], 
numpy.typing._array_like._SupportsArray[ForwardRef('dtype[Any]')], 
typing.Sequence[numpy.typing._array_like._SupportsArray[ForwardRef('dtype[Any]')]], 
typing.Sequence[typing.Sequence[numpy.typing._array_like._SupportsArray[
ForwardRef('dtype[Any]')]]], 
typing.Sequence[typing.Sequence[typing.Sequence[
numpy.typing._array_like._SupportsArray[ForwardRef('dtype[Any]')]]]], 
typing.Sequence[typing.Sequence[typing.Sequence[
typing.Sequence[numpy.typing._array_like._SupportsArray[
ForwardRef('dtype[Any]')]]]]], bool, int, float, complex, str, bytes, typing.Sequence[
typing.Union[bool, int, float, complex, str, bytes]], 
typing.Sequence[typing.Sequence[typing.Union[
bool, int, float, complex, str, bytes]]], typing.Sequence[typing.Sequence[typing.Sequence[
typing.Union[bool, int, float, complex, str, bytes]]]], 
typing.Sequence[typing.Sequence[typing.Sequence[
typing.Sequence[typing.Union[bool, int, float, complex, str, bytes]]]]]]
is not a generic class

Yikes. Let’s grep around the NumPy issue tracker and see if we can’t find anyone shouting vociferously about similar issues. That exception scares me like only grizzly bears snuffling through our compost pile in October can. 😮