question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

BUG: "replace" does not work with different numpy-int types even when values are the same

See original GitHub issue

Pandas version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • I have confirmed this bug exists on the master branch of pandas.

Reproducible Example

maps = pd.Series([np.int64(0),np.int64(2),np.int64(1)]) #index-value should be mapped to series-value

# 0 0    0-->0
# 1 2    1-->2
# 2 1    2-->1

map_dict = {old:new for (old,new) in zip(maps.values,maps.index)}

map_dict == {0: 0, 2: 1, 1: 2} #True

labs = pd.Series([1,1,1,0,0,2,2,2]).astype(np.int32) #Simulate a list of observations

(labs.replace({0: 0, 2: 1, 1: 2}) == labs.replace(map_dict)).mean() #0.25

labs.replace({0: 0, 2: 1, 1: 2}) # works

# 0    2
# 1    2
# 2    2
# 3    0
# 4    0
# 5    1
# 6    1
# 7    1



labs.replace(map_dict) #doesnt work
# 0    1
# 1    1
# 2    1
# 3    0
# 4    0
# 5    2
# 6    2
# 7    2

Issue Description

Using a mapping dictionary where the keys have np.int64 as type but a series of data where the values are np.int32 then the replacement is not working.

Expected Behavior

Expected to that all integer types can be evaluated against each-other

Installed Versions

INSTALLED VERSIONS

commit : 7c48ff4409c622c582c56a5702373f726de08e96 python : 3.8.1.final.0 python-bits : 64 OS : Windows OS-release : 10 Version : 10.0.19041 machine : AMD64 processor : Intel64 Family 6 Model 142 Stepping 10, GenuineIntel byteorder : little LC_ALL : None LANG : None LOCALE : Danish_Denmark.1252

pandas : 1.2.5 numpy : 1.21.0 pytz : 2021.1 dateutil : 2.8.1 pip : 19.2.3 setuptools : 41.2.0 Cython : 0.29.23 pytest : None hypothesis : None sphinx : None blosc : None feather : None xlsxwriter : None lxml.etree : None html5lib : None pymysql : None psycopg2 : 2.9.1 (dt dec pq3 ext lo64) jinja2 : None IPython : 7.24.1 pandas_datareader: None bs4 : 4.9.3 bottleneck : None fsspec : None fastparquet : None gcsfs : None matplotlib : 3.4.2 numexpr : 2.7.3 odfpy : None openpyxl : 3.0.7 pandas_gbq : None pyarrow : 5.0.0 pyxlsb : None s3fs : None scipy : 1.7.1 sqlalchemy : 1.4.20 tables : 3.6.1 tabulate : None xarray : None xlrd : None xlwt : None numba : None

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:9 (3 by maintainers)

github_iconTop GitHub Comments

1reaction
mroeschkecommented, Jan 14, 2022

Thanks for checking. Looks like a lot of bug fixes were applied to Series.replace in the upcoming version, 1.4, which may have fixed this issue. Don’t see this case explicitly mentions so it may be good to add a unit test

0reactions
vaish-mukcommented, Jan 27, 2022

take

Read more comments on GitHub >

github_iconTop Results From Across the Web

Pandas replace does not work with different numpy-int types ...
Pandas replace does not work with different numpy-int types even when values are the same · 1. Just a tip: map_dict = maps.to_dict()...
Read more >
NumPy 1.20.0 Release Notes
Replacing uses of items in the first column with the contents of the second column will work identically and silence the deprecation warning....
Read more >
Built-in Types — Python 3.11.1 documentation
Some operations are supported by several object types; in particular, ... The resultant value is a whole integer, though the result's type is...
Read more >
Chapter 4. NumPy Basics: Arrays and Vectorized Computation
Since NumPy is focused on numerical computing, the data type, if not ... Calling astype always creates a new array (a copy of...
Read more >
Handling Missing Data | Python Data Science Handbook
To make matters even more complicated, different data sources may indicate ... on the NumPy package, which does not have a built-in notion...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found