question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

REGR: FutureWarning issued and empty DataFrame returned where no numeric types to aggregate

See original GitHub issue

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • I have confirmed this bug exists on the master branch of pandas.

Reproducible Example

import numpy as np
import pandas as pd
frame = pd.DataFrame({"a": np.random.randint(0, 5, 50), "b": ["foo", "bar"] * 25})
frame[["b"]].groupby(frame["a"]).mean()

Issue Description

the code sample was raising DataError: No numeric types to aggregate in 1.2.5 but in 1.3.x this now issues a warning that it will raise in the future. (and also appears that the stacklevel is incorrect)

on master

>>> frame[["b"]].groupby(frame["a"]).mean()
sys:1: FutureWarning: Dropping invalid columns in DataFrameGroupBy.mean is deprecated. In a future version, a TypeError will be raised. Before calling .mean, select only columns which should be valid for the function.
Empty DataFrame
Columns: []
Index: [0, 1, 2, 3, 4]

The change to a empty dataframe was in #41706 with the warning being added in #43154

cc @jbrockmendel @Dr-Irv

Expected Behavior

same as 1.2.5

>>> frame[["b"]].groupby(frame["a"]).mean()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/simon/miniconda3/envs/pandas-1.2.5/lib/python3.9/site-packages/pandas/core/groupby/groupby.py", line 1496, in mean
    return self._cython_agg_general(
  File "/home/simon/miniconda3/envs/pandas-1.2.5/lib/python3.9/site-packages/pandas/core/groupby/generic.py", line 1015, in _cython_agg_general
    agg_mgr = self._cython_agg_blocks(
  File "/home/simon/miniconda3/envs/pandas-1.2.5/lib/python3.9/site-packages/pandas/core/groupby/generic.py", line 1121, in _cython_agg_blocks
    raise DataError("No numeric types to aggregate")
pandas.core.base.DataError: No numeric types to aggregate

Installed Versions

pd.show_versions()

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:14 (14 by maintainers)

github_iconTop GitHub Comments

1reaction
Dr-Irvcommented, Sep 11, 2021

@rshadrach so I’ve taken the table I made above and added a “Proposal” column, which I think is what you are suggesting above.

Code 1.2.5 Result 1.3.2 Result 1.4.0.dev0+647.g0072fa8cb5 Result Proposal
frame[["b"]].groupby(frame["a"]).mean() DataError exception Empty DataFrame FutureWarning and Empty DataFrame Empty DataFrame
frame[["b"]].groupby(frame["a"]).agg("mean") DataError exception Empty DataFrame FutureWarning and Empty DataFrame Empty DataFrame
frame[["b"]].groupby(frame["a"]).apply(np.mean) Empty DataFrame FutureWarning and Empty DataFrame FutureWarning and Empty DataFrame Empty DataFrame
frame[["b"]].groupby(frame["a"]).sum() DataFrame with values Empty DataFrame DataFrame with values DataFrame with values
frame[["b"]].groupby(frame["a"]).agg("sum") DataFrame with values Empty DataFrame DataFrame with values DataFrame with values
frame[["b"]].groupby(frame["a"]).apply(np.sum) DataFrame with values DataFrame with values DataFrame with values DataFrame with values

That would replicate the 1.2.5 behavior except where we were raising the DataError, replacing that with an empty DataFrame. It also seems that this would be the easiest fix to make??

0reactions
simonjayhawkinscommented, Nov 27, 2021

sure. closing as no action.

Read more comments on GitHub >

github_iconTop Results From Across the Web

No numeric types to aggregate - change in groupby ...
On 0.9, I get No numeric types to aggregate errors. Any ideas? In [31]: data Out[31]: <class 'pandas.core.frame.
Read more >
What's new in 1.4.0 (January 22, 2022) - Pandas
Previously, negative arguments returned empty frames. ... and Float64Index when given numeric data, but in the future, an Index will be returned.
Read more >
What's new in 1.1.0 (July 28, 2020) - Pandas
With groupby , we've added a dropna keyword to DataFrame.groupby() and Series.groupby() in order to allow NA values in group keys.
Read more >
What's new in 0.25.0 (July 18, 2019) - Pandas
The dtype of empty Index objects will now be evaluated before performing union operations rather than simply returning the other Index object. Index.union()...
Read more >
What's New — pandas 0.23.0 documentation - PyData |
Pandas now supports storing array-like objects that aren't necessarily 1-D NumPy arrays as columns in a DataFrame or values in a Series. This...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found