question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

ENH: enable skipna on groupby reduction ops

See original GitHub issue

https://github.com/pandas-dev/pandas/issues/15674

In [19]: import pandas as pd
    ...: import numpy as np
    ...: d = {'l':  ['left', 'right', 'left', 'right', 'left', 'right'],
    ...:      'r': ['right', 'left', 'right', 'left', 'right', 'left'],
    ...:      'v': [-1, 1, -1, 1, -1, np.nan]}
    ...: df = pd.DataFrame(d)
    ...: 

In [20]: df.groupby('l').v.sum()
Out[20]: 
l
left    -3.0
right    2.0
Name: v, dtype: float64

In [21]: df.groupby('l').v.apply(lambda x: x.sum(skipna=False))
Out[21]: 
l
left    -3.0
right    NaN
Name: v, dtype: float64

ideally write [21] as df.groupby('l').v.sum(skipna=False)

Issue Analytics

  • State:open
  • Created 7 years ago
  • Reactions:8
  • Comments:8 (4 by maintainers)

github_iconTop GitHub Comments

2reactions
vladucommented, Jun 29, 2022

Indeed, I believe this is not just a “nice-to-have” any more, but a necessary step before this FutureWarning can be resolved by users:

FutureWarning: Using the level keyword in DataFrame and Series aggregations is deprecated and will be removed in a future version. Use groupby instead. df.sum(level=1) should use df.groupby(level=1).sum().

As long as DataFrame.sum() and DataFrame.groupby().sum() (and other agg functions) have inconsistent APIs, dropping the level kwarg from non-grouped classes isn’t really a good step, IMO.

2reactions
mullenkampcommented, May 31, 2022

This is a surprisingly old issue, but this functionality would be really nice and consistent with the non-groupby methods.

Read more comments on GitHub >

github_iconTop Results From Across the Web

pandas GroupBy columns with NaN (missing) values
see that Pandas has dropped the rows with NaN target values. (I want to include these rows!) Since I need many such operations...
Read more >
What's new in 1.4.0 (January 22, 2022) - Pandas
GroupBy.cummin() and GroupBy.cummax() now support the argument skipna (GH34047) ... Reduction operations for DataFrame or Series now raising a ValueError ...
Read more >
Shuffling for GroupBy and Join - Dask documentation
To start off, common groupby operations like df.groupby(columns).reduction() for known reductions like mean, sum, std, var, count, nunique are all quite ...
Read more >
GroupBy: Group and Bin Data - Xarray
To do this, Xarray supports “group by” operations with the same API as pan... ... variance) on the groups, and then return a...
Read more >
Pandas GroupBy: Group, Summarize, and Aggregate Data in ...
The Pandas .groupby() method works in a very similar way to the SQL GROUP BY ... This allows you to perform operations on...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found