datetime64 arrays don't support buffer protocol

In [4] m = memoryview(np.array(['2001-01-01'], dtype='M8[D]'))
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-4-033a8bae908a> in <module>()
----> 1 m = memoryview(np.array(['2001-01-01'], dtype='M8[D]'))

ValueError: cannot include dtype 'M' in a buffer

[abalkin]

I suspect the problem is at Objects/memoryobject.c:731,

   (void)PyBuffer_FillInfo(&mbuf->master, NULL, mem, size, readonly,
                            PyBUF_FULL_RO);

where the return value of PyBuffer_FillInfo is ignored. I’ll submit an upstream issue.

[eric-wieser]

An idea I had about this (which I might have put elsewhere) was to embed numpy type data in the existing buffer spec by exploiting the fact that T{} uses no storage space.

def _as_buffer(arr):
    try:
        return memoryview(arr)
    except ValueError:
        dt = arr.dtype
        new_dt = np.dtype([
            ('__numpy_value', (np.void, dt.itemsize)),
            ('__numpy_dtype', [
                (str(dt), [])
            ])
        ])
        return memoryview(arr.view(new_dt))

def _from_buffer(b):
    arr = np.asarray(b)
    if np.issubdtype(arr.dtype, np.void) and arr.dtype.names == ('__numpy_value', '__numpy_dtype'):
        new_dtype_name, = arr.dtype['__numpy_dtype'].names
        return arr['__numpy_value'].view(new_dtype_name)
    else:
        return arr

Which works as

>>> a = np.array(['now', 'now'], np.datetime64) + [0, 1]
>>> buffer_a = _as_bufferable(a)
>>> buffer_a.format
'T{8x:__numpy_value:T{T{}:datetime64[s]:}:__numpy_dtype:}'

---

A scarier approach might be to use pickle.dumps(arr.dtype).encode('hex') as the field name, to ensure that absolutely any user dtype can be used.