Support structured arrays
See original GitHub issueIt looks like cupy supports record arrays, but fails to accept a list dtype that was not properly packed as a dtype object. My guess is that this could
In [1]: import numpy as np
In [2]: import cupy
In [3]: x = cupy.empty(shape=5, dtype=np.dtype([('a', int), ('b', float)]))
In [4]: x
Out[4]:
array([(0, 0.), (0, 0.), (0, 0.), (0, 0.), (0, 0.)],
dtype=[('a', '<i8'), ('b', '<f8')])
In [5]: x = cupy.empty(shape=5, dtype=[('a', int), ('b', float)])
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-5-8478cd56259b> in <module>
----> 1 x = cupy.empty(shape=5, dtype=[('a', int), ('b', float)])
~/cupy/cupy/creation/basic.py in empty(shape, dtype, order)
20
21 """
---> 22 return cupy.ndarray(shape, dtype, order=order)
23
24
~/cupy/cupy/core/core.pyx in cupy.core.core.ndarray.__init__()
~/cupy/cupy/core/_dtype.pyx in cupy.core._dtype.get_dtype_with_itemsize()
TypeError: unhashable type: 'list'
Numpy handles this ok. My guess is that this could be resolved by a quick typecheck, or just always calling np.dtype
on the input to any dtype=
keyword.
Issue Analytics
- State:
- Created 5 years ago
- Comments:17 (11 by maintainers)
Top Results From Across the Web
Structured arrays — NumPy v1.24 Manual
Structured arrays are ndarrays whose datatype is a composition of simpler ... For these purposes they support specialized features such as subarrays, ...
Read more >Structure array - MATLAB
A structure array is a data type that groups related data using data containers called fields. Each field can contain any type of...
Read more >AoS and SoA
In computing, array of structures (AoS), structure of arrays (SoA) and array of structures of arrays (AoSoA) refer to contrasting ways to arrange...
Read more >Structure arrays
Restrictions with the structure array support · Structure arrays are supported by C or C++ embedded SQL applications that connect to DB2® for...
Read more >Structured arrays with nd-fields - Support - Numba Discussion
Also, a nested array is guaranteed to be contiguous in memory with the rest of the struct, while a namedtuple of arrays will...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Perfect. Thanks
Sorry, I switched to a simpler SbtRecord layout for my 2nd example code with just the header and a float3 for the user data.
In a real-world case, the struct’s post-header user data is likely to be much more complicated and heterogenous. I worry about your approach scaling with complexity. Here is my first example again:
It is more difficult to see how to set the data for an array of structs of this type. The layout of the data needs to match (with padding between fields) the layout on the C side and the array could have 100,000+ elements. The structured array in numpy is designed for this.
One simplifying assumption is that I never need to perform any cupy operations on the array. once I pass it to cupy it is just a data blob to me – I only want cupy to copy the bytes to the device to get a device address for where it exists on the GPU. any operations on the device-side data will occur in optix or in a raw cuda kernel. so this has me thinking, would it make more sense to just use numpy to allocate and populate a structured array, get the host-side C pointer from the numpy array (if this is possible), and then use
cupy.cuda.runtime.memcpyAsync
to just transfer the bytes from host to device?