question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

BUG: Replace methods fills value from previous row when replacing with None

See original GitHub issue
  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • (optional) I have confirmed this bug exists on the master branch of pandas.


Code Sample, a copy-pastable example

import numpy as np
import pandas as pd

df = pd.DataFrame({'x': [10, 20, np.nan], 'y': [30, 40, 50]})
print(df.replace(np.NaN, None))

#       x   y
# 0  10.0  30
# 1  20.0  40
# 2  20.0  50

Problem description

When replacing NaN with None using replace method, value, NaN value is replaced with value from previous row instead of None.

Expected Output

#       x   y
# 0  10.0  30
# 1  20.0  40
# 2  None  50

Output of pd.show_versions()

INSTALLED VERSIONS
------------------
commit           : None
python           : 3.7.4.final.0
python-bits      : 64
OS               : Windows
OS-release       : 10
machine          : AMD64
processor        : Intel64 Family 6 Model 158 Stepping 13, GenuineIntel
byteorder        : little
LC_ALL           : None
LANG             : en_US.UTF-8
LOCALE           : None.None
pandas           : 1.0.0rc0
numpy            : 1.17.2
pytz             : 2019.2
dateutil         : 2.8.0
pip              : 19.3.1
setuptools       : 40.8.0
Cython           : None
pytest           : None
hypothesis       : None
sphinx           : None
blosc            : None
feather          : None
xlsxwriter       : None
lxml.etree       : None
html5lib         : None
pymysql          : None
psycopg2         : None
jinja2           : None
IPython          : None
pandas_datareader: None
bs4              : None
bottleneck       : None
fastparquet      : None
gcsfs            : None
lxml.etree       : None
matplotlib       : None
numexpr          : None
odfpy            : None
openpyxl         : None
pandas_gbq       : None
pyarrow          : None
pytables         : None
pytest           : None
s3fs             : None
scipy            : None
sqlalchemy       : None
tables           : None
tabulate         : 0.8.6
xarray           : None
xlrd             : None
xlwt             : None
xlsxwriter       : None
numba            : None

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:5 (4 by maintainers)

github_iconTop GitHub Comments

1reaction
DataSolveProblemscommented, Apr 5, 2020

I was able to workaround by passing a dictionary. Not ideal, but works.

print(df.replace({np.NaN: None}))
0reactions
simonjayhawkinscommented, Apr 5, 2020

from the examples in https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.replace.html, it looks like this maybe expected behaviour as the default value for method is pad

This issue also looks like a duplicate of #19998, so closing. ping to reopen if I missed something.

Read more comments on GitHub >

github_iconTop Results From Across the Web

python 3.x - Replacing NaN value with None fill the value from ...
When I use DataFrame.replace() method to replace NaN with None, dataframe is taking value from previous row instead of None value.
Read more >
Pandas Replace Values- pd.DataFrame.replace()
Pandas Replace - .replace() will find values within your Pandas DataFrame, then replace with new values. This function starts simple, but is super...
Read more >
How to fill empty cells with previous row value in Excel
Have a lot of gaps in your data that have to be filled with the data above? Learn how to quickly & easily...
Read more >
Excel: find and replace multiple values at once - Ablebits
The easiest way to find and replace multiple entries in Excel is by using the SUBSTITUTE function. The formula's logic is very simple:...
Read more >
Replace values and errors - Power Query - Microsoft Learn
On the cell shortcut menu. Right-click the cell to replace the selected value in the column with another value. Shortcut menu for replacing...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found