question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Inconsistent behavior of dtype.descr with metadata

See original GitHub issue

We used to the following property: if x is a dtype then numpy.dtype(x.descr) is the same dtype. This property is broken in NumPy 1.8 when one specifies metadata to dtype. Is it possible to at least fix that descr returned by dtype can always be evaluated back to a dtype?

Related issues: https://github.com/h5py/h5py/issues/1307

Reproducing code example:

import numpy as np

# 1. metadata is lost for simple dtype
s_dt = np.dtype('S8', metadata={'msg': 'Hello'})
print(s_dt.descr)
# prints [('', '|S8')]

# 2. metadata is returned for structured dtype, but it cannot be evaluated into dtype
struct_dtype = np.dtype([('a', s_dt)])
print(struct_dtype.descr)
# prints [('a', ('|S8', {'msg': 'Hello'}))]

# next line raises a ValueError exception
# ValueError: invalid shape in fixed-type tuple.
np.dtype(struct_dtype.descr)

Numpy/Python version information:

1.18.1 3.8.1 | packaged by conda-forge | (default, Jan 29 2020, 14:55:04) [GCC 7.3.0]

Issue Analytics

  • State:open
  • Created 4 years ago
  • Comments:21 (13 by maintainers)

github_iconTop GitHub Comments

1reaction
mattipcommented, Feb 6, 2020

I am not sure what descr is actually good for

Maybe deprecate/make it private as part of the dtype refactor?

1reaction
sebergcommented, Feb 5, 2020

I am not sure what descr is actually good for 😃. We postprocess in a few places, but other than that? Trying to cram this feature in descr is just a bad idea. And it may well be that nobody really should be using .descr at all?

Read more comments on GitHub >

github_iconTop Results From Across the Web

numpy.dtype.descr — NumPy v1.24 Manual
Warning: This attribute exists specifically for __array_interface__, and passing it directly to np.dtype will not accurately reconstruct some dtypes (e.g., ...
Read more >
What's new in 1.5.0 (September 19, 2022) - Pandas
With Pyarrow installed, users can now create pandas objects that are backed by a pyarrow.ChunkedArray and pyarrow.DataType . The dtype argument can accept...
Read more >
Drop in data consistency - IBM Cloud Pak for Data as a Service
If a transaction violates one or more of the training data patterns, the transaction is identified as inconsistent. To calculate the drop in ......
Read more >
Finding invalid values in numerical columns | Drawing from Data
Specify dtype option on import or set low_memory=False. ... Object Wikidata URL https://www.wikidata.org/wiki/Q83545838 Metadata Date NaN ...
Read more >
Release Notes — NumPy v1.16 Manual
numpy.ndarray.getfield now checks the dtype and offset arguments to prevent ... which is inconsistent with the behavior for float inputs, ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found