BUG: Replace methods fills value from previous row when replacing with None
See original GitHub issue-
I have checked that this issue has not already been reported.
-
I have confirmed this bug exists on the latest version of pandas.
-
(optional) I have confirmed this bug exists on the master branch of pandas.
Code Sample, a copy-pastable example
import numpy as np
import pandas as pd
df = pd.DataFrame({'x': [10, 20, np.nan], 'y': [30, 40, 50]})
print(df.replace(np.NaN, None))
# x y
# 0 10.0 30
# 1 20.0 40
# 2 20.0 50
Problem description
When replacing NaN with None using replace method, value, NaN value is replaced with value from previous row instead of None.
Expected Output
# x y
# 0 10.0 30
# 1 20.0 40
# 2 None 50
Output of pd.show_versions()
INSTALLED VERSIONS
------------------
commit : None
python : 3.7.4.final.0
python-bits : 64
OS : Windows
OS-release : 10
machine : AMD64
processor : Intel64 Family 6 Model 158 Stepping 13, GenuineIntel
byteorder : little
LC_ALL : None
LANG : en_US.UTF-8
LOCALE : None.None
pandas : 1.0.0rc0
numpy : 1.17.2
pytz : 2019.2
dateutil : 2.8.0
pip : 19.3.1
setuptools : 40.8.0
Cython : None
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : None
IPython : None
pandas_datareader: None
bs4 : None
bottleneck : None
fastparquet : None
gcsfs : None
lxml.etree : None
matplotlib : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pytables : None
pytest : None
s3fs : None
scipy : None
sqlalchemy : None
tables : None
tabulate : 0.8.6
xarray : None
xlrd : None
xlwt : None
xlsxwriter : None
numba : None
Issue Analytics
- State:
- Created 3 years ago
- Comments:5 (4 by maintainers)
Top Results From Across the Web
python 3.x - Replacing NaN value with None fill the value from ...
When I use DataFrame.replace() method to replace NaN with None, dataframe is taking value from previous row instead of None value.
Read more >Pandas Replace Values- pd.DataFrame.replace()
Pandas Replace - .replace() will find values within your Pandas DataFrame, then replace with new values. This function starts simple, but is super...
Read more >How to fill empty cells with previous row value in Excel
Have a lot of gaps in your data that have to be filled with the data above? Learn how to quickly & easily...
Read more >Excel: find and replace multiple values at once - Ablebits
The easiest way to find and replace multiple entries in Excel is by using the SUBSTITUTE function. The formula's logic is very simple:...
Read more >Replace values and errors - Power Query - Microsoft Learn
On the cell shortcut menu. Right-click the cell to replace the selected value in the column with another value. Shortcut menu for replacing...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
I was able to workaround by passing a dictionary. Not ideal, but works.
from the examples in https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.replace.html, it looks like this maybe expected behaviour as the default value for
method
ispad
This issue also looks like a duplicate of #19998, so closing. ping to reopen if I missed something.