ENH: enable skipna on groupby reduction ops
See original GitHub issuehttps://github.com/pandas-dev/pandas/issues/15674
In [19]: import pandas as pd
...: import numpy as np
...: d = {'l': ['left', 'right', 'left', 'right', 'left', 'right'],
...: 'r': ['right', 'left', 'right', 'left', 'right', 'left'],
...: 'v': [-1, 1, -1, 1, -1, np.nan]}
...: df = pd.DataFrame(d)
...:
In [20]: df.groupby('l').v.sum()
Out[20]:
l
left -3.0
right 2.0
Name: v, dtype: float64
In [21]: df.groupby('l').v.apply(lambda x: x.sum(skipna=False))
Out[21]:
l
left -3.0
right NaN
Name: v, dtype: float64
ideally write [21] as
df.groupby('l').v.sum(skipna=False)
Issue Analytics
- State:
- Created 7 years ago
- Reactions:8
- Comments:8 (4 by maintainers)
Top Results From Across the Web
pandas GroupBy columns with NaN (missing) values
see that Pandas has dropped the rows with NaN target values. (I want to include these rows!) Since I need many such operations...
Read more >What's new in 1.4.0 (January 22, 2022) - Pandas
GroupBy.cummin() and GroupBy.cummax() now support the argument skipna (GH34047) ... Reduction operations for DataFrame or Series now raising a ValueError ...
Read more >Shuffling for GroupBy and Join - Dask documentation
To start off, common groupby operations like df.groupby(columns).reduction() for known reductions like mean, sum, std, var, count, nunique are all quite ...
Read more >GroupBy: Group and Bin Data - Xarray
To do this, Xarray supports “group by” operations with the same API as pan... ... variance) on the groups, and then return a...
Read more >Pandas GroupBy: Group, Summarize, and Aggregate Data in ...
The Pandas .groupby() method works in a very similar way to the SQL GROUP BY ... This allows you to perform operations on...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
Indeed, I believe this is not just a “nice-to-have” any more, but a necessary step before this
FutureWarning
can be resolved by users:As long as
DataFrame.sum()
andDataFrame.groupby().sum()
(and other agg functions) have inconsistent APIs, dropping thelevel
kwarg from non-grouped classes isn’t really a good step, IMO.This is a surprisingly old issue, but this functionality would be really nice and consistent with the non-groupby methods.