question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

BUG: Series.fillna raising with `float32` dtype when using value arg

See original GitHub issue
  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • (optional) I have confirmed this bug exists on the master branch of pandas.


Code Sample, a copy-pastable example

ser = pd.Series([1.0, 2.0], dtype="float32")
val = {1: 1.01}
ser.fillna(val)

Problem description

The above code raises with TypeError: Cannot cast array data from dtype('float64') to dtype('float32') according to the rule 'safe'

Expected Output

0    1.0
1    2.0
dtype: float32

Output of pd.show_versions()

INSTALLED VERSIONS

commit : 343ac2a4179fdd1cb24633e9b95cbec33371c14d python : 3.8.10.final.0 python-bits : 64 OS : Darwin OS-release : 20.4.0 Version : Darwin Kernel Version 20.4.0: Thu Apr 22 21:46:47 PDT 2021; root:xnu-7195.101.2~1/RELEASE_X86_64 machine : x86_64 processor : i386 byteorder : little LC_ALL : None LANG : None LOCALE : None.UTF-8

pandas : 1.4.0.dev0+599.g343ac2a417.dirty numpy : 1.21.1 pytz : 2021.1 dateutil : 2.8.2 pip : 21.2.1 setuptools : 49.6.0.post20210108 Cython : 0.29.24 pytest : 6.2.4 hypothesis : 6.14.3 sphinx : 3.5.4 blosc : None feather : None xlsxwriter : 1.4.4 lxml.etree : 4.6.3 html5lib : 1.1 pymysql : None psycopg2 : None jinja2 : 3.0.1 IPython : 7.25.0 pandas_datareader: None bs4 : 4.9.3 bottleneck : 1.3.2 fsspec : 2021.05.0 fastparquet : 0.6.3 gcsfs : 2021.05.0 matplotlib : 3.4.2 numexpr : 2.7.3 odfpy : None openpyxl : 3.0.7 pandas_gbq : None pyarrow : 4.0.1 pyxlsb : None s3fs : 0.4.2 scipy : 1.7.0 sqlalchemy : 1.4.22 tables : 3.6.1 tabulate : 0.8.9 xarray : 0.19.0 xlrd : 2.0.1 xlwt : 1.3.0 numba : 0.53.1

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:8 (8 by maintainers)

github_iconTop GitHub Comments

1reaction
simonjayhawkinscommented, Sep 7, 2021

A nice patch would seem to be

my comment was more to indicate that the issue is a regression rather than suggesting that the old code is a better fix. Not all users upgrade on every pandas release, so if this is fixed, we could backport and may save some users some pain. (but have not milestoned 1.3.x since it is an “old” regression)

0reactions
jbrockmendelcommented, Sep 7, 2021

Not sure if I follow the whole thread, but if changing a np.putmask usage to a __setitem__ usage fixes it, thats fine. We made an effort to use np.putmask where possible a while back bc it improved perf in dtlike comparisons

Read more comments on GitHub >

github_iconTop Results From Across the Web

[Bug]: `plt.plot` thinks `pandas.Series` is 2-dimensional when ...
I think I agree, I think they are raising a different exception than we are expecting and inconsistent by dtype. Opening a bug...
Read more >
Data type conversion error: ValueError: Cannot convert non ...
I am using the following code to convert "birth year" column type from float to int. df1[['birth year']] = df1[['birth year']].astype(int) print ...
Read more >
What's new in 1.4.0 (January 22, 2022) - Pandas
Bug in setting dtype-incompatible values into a Categorical (or Series or DataFrame backed by Categorical ) raising ValueError instead of TypeError (GH41919).
Read more >
pandas-docs-zh
Bug in .fillna(value=np.nan) incorrectly raises KeyError on a category dtyped Series (:is- sue:'14021'). • Bug in extension dtype creation where the created ...
Read more >
What's New — pandas 0.23.4 documentation
Bug in read_csv() that caused it to incorrectly raise an error when nrows=0 ... Series.fillna() now accepts a Series or a dict as...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found