question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

BUG: DataFrame.replace fails to replace value when column contains pd.NA

See original GitHub issue

Pandas version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd

df = pd.DataFrame({'A': [0, 1, 2]})
print(df)
#    A
# 0  0
# 1  1
# 2  2
df['A'].replace(to_replace=2, value=99, inplace=True)
print(df)
#     A
# 0   0
# 1   1
# 2  99
df.at[0, 'A'] = pd.NA
df['A'].replace(to_replace=1, value=100, inplace=True)
print(df)
#       A
# 0  <NA>
# 1     1  <-- should be 100
# 2    99

Issue Description

Pandas replace function does not seem to work on a column if the column contains at least one pd.NA value

Expected Behavior

replace function should work even if pd.NA values are in the column

Installed Versions

INSTALLED VERSIONS

commit : 66e3805b8cabe977f40c05259cc3fcf7ead5687d python : 3.10.0.final.0 python-bits : 64 OS : Linux OS-release : 5.16.19-76051619-generic Version : #202204081339~1649696161~20.04~091f44b~dev-Ubuntu SMP PREEMPT Tu machine : x86_64 processor : x86_64 byteorder : little LC_ALL : None LANG : en_US.UTF-8 LOCALE : en_US.UTF-8

pandas : 1.3.5 numpy : 1.21.2 pytz : 2021.3 dateutil : 2.8.2 pip : 21.2.4 setuptools : 58.0.4 Cython : None pytest : None hypothesis : None sphinx : None blosc : None feather : None xlsxwriter : 3.0.3 lxml.etree : None html5lib : None pymysql : None psycopg2 : None jinja2 : 3.0.2 IPython : 7.29.0 pandas_datareader: None bs4 : None bottleneck : None fsspec : None fastparquet : None gcsfs : None matplotlib : 3.5.1 numexpr : None odfpy : None openpyxl : 3.0.9 pandas_gbq : None pyarrow : None pyxlsb : None s3fs : None scipy : 1.8.0 sqlalchemy : None tables : None tabulate : None xarray : None xlrd : None xlwt : None numba : None

Issue Analytics

  • State:open
  • Created a year ago
  • Comments:19 (11 by maintainers)

github_iconTop GitHub Comments

2reactions
phoflcommented, Jul 13, 2022

NA = NA is again NA, this happens on purpose

1reaction
phoflcommented, Aug 8, 2022
Read more comments on GitHub >

github_iconTop Results From Across the Web

DataFrame.replace fails to replace value when columns are ...
DataFrame.replace fails to replace value when columns are specified and only non-replacement columns contain pd.NA #32838.
Read more >
replace() method not working on Pandas DataFrame
The aim is to replace a string anywhere in the dataframe with an nan, however this does not seem to work (i.e. does...
Read more >
Pandas Replace Values- pd.DataFrame.replace()
Pandas Replace - .replace() will find values within your Pandas DataFrame, then replace with new values. This function starts simple, but is super...
Read more >
Working with missing data - Pandas
Because NaN is a float, a column of integers with even one missing values is cast ... Series([1, 2, np.nan, 4], dtype=pd. ......
Read more >
Working with missing data — pandas 1.2.0 documentation
Because NaN is a float, a column of integers with even one missing values is cast to ... Replace NA with a scalar...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found