Support duck arrays in numba gufuncs
See original GitHub issueThere might be some room for improvement in handling numpy-like arrays in numba-generated gufuncs
I was quite happy to see this work well:
In [1]: from numba import vectorize, float64
...:
...: @vectorize([float64(float64, float64)])
...: def f(x, y):
...: return x + y
...:
In [2]: import dask.array as da
In [3]: x = da.arange(10, chunks=(5,))
In [4]: f(x, x) # hooray!
Out[4]: dask.array<f, shape=(10,), dtype=float64, chunksize=(5,)>
Unfortunately this didn’t fare as well. Am I using vectorize incorrectly or is this a feature request?
In [5]: @vectorize
...: def f(x, y):
...: return x + y
...:
In [6]: f(x, x) # oh no!
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-6-4438cca8adef> in <module>()
----> 1 f(x, x) # oh no!
~/Software/anaconda/lib/python3.6/site-packages/numba/npyufunc/dufunc.py in _compile_for_args(self, *args, **kws)
162 argtys.append(numpy_support.map_arrayscalar_type(arg))
163 else:
--> 164 argty = typeof(arg)
165 if isinstance(argty, types.Array):
166 argty = argty.dtype
~/Software/anaconda/lib/python3.6/site-packages/numba/typing/typeof.py in typeof(val, purpose)
29 if ty is None:
30 msg = "cannot determine Numba type of %r" % (type(val),)
---> 31 raise ValueError(msg)
32 return ty
33
ValueError: cannot determine Numba type of <class 'dask.array.core.Array'>
Issue Analytics
- State:
- Created 5 years ago
- Comments:7 (4 by maintainers)
Top Results From Across the Web
Numpy Support in numba — numba 0.16.0 documentation
Access to Numpy arrays is very efficient, as indexing is lowered to memory accessing when possible. * numba is able to generate ufuncs/gufuncs....
Read more >stable PDF - Numba Documentation
Numba is a just-in-time compiler for Python that works best on code that uses NumPy arrays and functions, and loops.
Read more >Numba for CUDA Programmers - GitHub
What is Numba? A Just-in-time (JIT) compiler for Python functions. ▻ Opt-in: Numba only compiles the functions you specify. ▻ Focused on array-oriented ......
Read more >Expansion of generalized universal function signatures - NumPy
Possibly missing dimensions. This part is almost entirely driven by the wish to wrap matmul in a gufunc. matmul stands for matrix multiplication ......
Read more >Applying unvectorized functions with apply_ufunc - Xarray
The function we will apply is np.interp which expects 1D numpy arrays. ... 797 "applied function returned data with unexpected " 798 f"number...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Just to put a bit of motivation behind this feature request, I’d like to write a blogpost that shows that numba and dask.array are able to inter-operate over the gufunc protocol without explicit coordination.
@mrocklin, yes, your comment in https://github.com/sklam/numba/commit/34f9d48b10603948b1b345091b4d8175217614bc is exactly right. I have only done the minimal to see how to make it work and to see what code needed to be changed. If someone wanted to take on the work, they would need to:
dask.array
; using some made up type with__array_ufunc__
).