Why pd.BooleanDtype() is casted to Float64 by groupby/last?
See original GitHub issueCode Sample, a copy-pastable example if possible
>>> import pandas as pd
>>>
>>> df = pd.DataFrame({'a': ['x', 'x', 'y', 'y'], 'b': ['x', 'x', 'y', 'y'], 'c': [False, False, True, False]})
>>> df['d'] = df.c.astype(pd.BooleanDtype())
>>>
>>> df.dtypes
a object
b object
c bool
d boolean
dtype: object
>>>
>>> df.groupby(['a', 'b']).c.last()
a b
x x False
y y False
Name: c, dtype: bool
>>>
>>> df.groupby(['a', 'b']).d.last()
a b
x x 0.0
y y 0.0
Name: d, dtype: float64
>
Problem description
df.groupby([‘a’, ‘b’]).c.last() returns False, but df.groupby([‘a’, ‘b’]).d.last() returns Float64. Why the difference?
Expected Output
I expect that both values should be False
Output of pd.show_versions()
python : 3.7.4.final.0 pandas : 1.0.3
Issue Analytics
- State:
- Created 3 years ago
- Comments:9 (5 by maintainers)
Top Results From Across the Web
unable to change from object type to float64 in pandas
It is complex to replicate my df with the dtypes of the columns, but I will post the df used for this case....
Read more >What's new in 0.24.0 (January 25, 2019) - Pandas
A newly constructed empty DataFrame with integer as the dtype will now only be cast to float64 if index is specified (GH22858) ;...
Read more >CVS log for pkgsrc/math/py-pandas/PLIST - NetBSD CVS Repositories
Fixed regressions * Fixed regression in DataFrame.convert_dtypes() incorrectly ... a dtype which couldn't cast lostlessly (like float32 filled with float64) ...
Read more >What's new in 0.24.0 (January 25, 2019) | Pandas - Pandas 中文
These dtypes can be merged, reshaped, and casted. In [11]: pd.concat([df[['A']], df[['B', 'C']]], axis=1).dtypes Out[11]: A Int64 B int64 C object Length: 3 ......
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
You are welcome. I am glad that I can participate (at least by testing) to the development of such a marvel as pandas is.
Best regards.
@ghuname We can leave this issue open, it should be closed automatically if / when the associated PR is merged