Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Pickling `type` of `dtype` object fails on NumPy 1.20

See original GitHub issue

Reproducing code example:

On NumPy 1.19.5, the following is possible…

In [1]: import pickle
   ...: import numpy as np

In [2]: t = type(np.arange(10, dtype="int64").dtype)

In [3]: pickle.loads(pickle.dumps(t))
Out[3]: numpy.dtype

However on NumPy 1.20.0 we see the following error…

In [1]: import pickle
   ...: import numpy as np

In [2]: t = type(np.arange(10, dtype="int64").dtype)

In [3]: pickle.loads(pickle.dumps(t))
---------------------------------------------------------------------------
PicklingError                             Traceback (most recent call last)
<ipython-input-3-c105e6dd98fb> in <module>
----> 1 pickle.loads(pickle.dumps(t))

PicklingError: Can't pickle <class 'numpy.dtype[int64]'>: attribute lookup dtype[int64] on numpy failed

Not sure if this is expected to fail, but we encountered this in our test suite when upgrading to NumPy 1.20

Error message:

---------------------------------------------------------------------------
PicklingError                             Traceback (most recent call last)
<ipython-input-3-c105e6dd98fb> in <module>
----> 1 pickle.loads(pickle.dumps(t))

PicklingError: Can't pickle <class 'numpy.dtype[int64]'>: attribute lookup dtype[int64] on numpy failed

NumPy/Python version information:

1.20.0 3.9.1 | packaged by conda-forge | (default, Jan 26 2021, 01:32:59) 
[Clang 11.0.1 ]

cc @madsbk @pentschev @quasiben

Issue Analytics

State:
Created 3 years ago
Reactions:1
Comments:24 (14 by maintainers)

Top GitHub Comments

1reaction

sebergcommented, Feb 5, 2021

Yeah, pickling the dtype.type is totally fine (in fact, it is the solution here at the moment). dtype.type is the scalar type though which is different from type(dtype) its a bit confusing 😃.

1reaction

sebergcommented, Feb 4, 2021

No need to rebuild, its just a pure python change. Thanks!

Top Results From Across the Web

UMAP PicklingError: ("Can't pickle <class 'numpy.dtype[float32]'>

The problem seems to be with Numpy. I was running 1.20 when hitting this error. Downgrading with pip install numpy==1.19. resolves it.

Data type objects (dtype) — NumPy v1.24 Manual

A data type object (an instance of numpy.dtype class) describes how the bytes in the fixed-size block of memory corresponding to an array...

Release notes — NumPy v1.24 Manual

1D np.linalg.norm preserves float input types, even for scalar results · Changes to structured (void) dtype promotion and comparisons ...

NumPy 1.20.0 Release Notes

Code that may have used type(dtype) is np.dtype will always return False and must be updated to use the correct version isinstance(dtype, np.dtype)...

numpy.load — NumPy v1.24 Manual

Loading files that contain object arrays uses the pickle module, which is not secure against erroneous or maliciously constructed data. Consider passing ...