Raise `NotImplemented` in binary operations with non-CuPy arrays

Binary operations in CuPy currently raise TypeError when other is a non-CuPy array, see below:

In [1]: import cupy as cp, dask.array as da

In [2]: a = cp.arange(5)

In [3]: b = da.ones_like(a, shape=(5,))

In [4]: b - a
Out[4]: dask.array<sub, shape=(5,), dtype=int64, chunksize=(5,), chunktype=cupy.ndarray>

In [5]: a - b
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-5-09bd029d0285> in <module>
----> 1 a - b

cupy/core/core.pyx in cupy.core.core.ndarray.__sub__()

cupy/core/_kernel.pyx in cupy.core._kernel.ufunc.__call__()

cupy/core/_kernel.pyx in cupy.core._kernel._preprocess_args()

TypeError: Unsupported type <class 'dask.array.core.Array'>

It would be useful to instead raise a NotImplemented exception as it would allow the other array to try to compute that operation, see NotImplemented docs for details.

Any thoughts on allowing this in CuPy?

[seberg]

From my perspective, array-priority is more of an artifact of the past. We probably can’t quite do without it yet, but I think we should try.

Normally, the logic in __add__ is something like:

def __add__(self, other):
    if not is_known_type(other):
        return NotImplemented

NumPy (and cupy probably) can’t do that easily, because we coerce most python objects agressively. The solution is to use __array_ufunc__ to decide whether this is a “coercible” type:

def __add__(self, other):
    if hasattr(other, "__array_ufunc__") and not is_known_type(other):
        return NotImplemented

NumPy continues to check the array priority for historic reasons IMO. And I am not even sure cupy/dask have to worry about it?

In the previous post I thought you may need a 3-way logic (calling the ufunc in theory). I guess that should not be necessary, if self.__array_ufunc__ is valid, is_known_type() will succeed since you use the same logic for __array_ufunc__. If other.__array_ufunc__ would be used, then other.__add__ should succeed as well and deferring is correct.

Or am I missing a reason why cupy/dask must rely on __array_priority__ here?

[asi1024]

@pentschev @mrocklin @seberg Thanks for your explanations! I fixed cupy.ndarray binary operation logic in #4198.