Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

BUG: `lambda` with positional references in `apply` after `groupby` on empty dataframe errors

See original GitHub issue

Pandas version checks

I have checked that this issue has not already been reported.
I have confirmed this bug exists on the latest version of pandas.
I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

>>> import pandas as pd
>>> pd.DataFrame([],columns=['a','b','c']).groupby('a').b.apply(sum)
Series([], Name: b, dtype: object)
>>> pd.Series([],dtype='float64').apply(sum)
Series([], dtype: float64)
>>> pd.Series([],dtype='float64').apply(lambda x: x[0]-x[1])
Series([], dtype: float64)
>>> pd.DataFrame([],columns=['a','b','c']).groupby('a').b.apply(lambda x: x[0]-x[1])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/Jesse/repo/frame/dev-virtualenv-frame/lib/python3.9/site-packages/pandas/core/groupby/generic.py", line 244, in apply
    return super().apply(func, *args, **kwargs)
  File "/Users/Jesse/repo/frame/dev-virtualenv-frame/lib/python3.9/site-packages/pandas/core/groupby/groupby.py", line 1414, in apply
    result = self._python_apply_general(f, self._selected_obj)
  File "/Users/Jesse/repo/frame/dev-virtualenv-frame/lib/python3.9/site-packages/pandas/core/groupby/groupby.py", line 1455, in _python_apply_general
    values, mutated = self.grouper.apply(f, data, self.axis)
  File "/Users/Jesse/repo/frame/dev-virtualenv-frame/lib/python3.9/site-packages/pandas/core/groupby/ops.py", line 776, in apply
    f(data.iloc[:0])
  File "<stdin>", line 1, in <lambda>
  File "/Users/Jesse/repo/frame/dev-virtualenv-frame/lib/python3.9/site-packages/pandas/core/series.py", line 955, in __getitem__
    return self._values[key]
IndexError: index 0 is out of bounds for axis 0 with size 0

Issue Description

apply on empty series after groupby attempts to evaluate and errors

Expected Behavior

returns empty series

Installed Versions

{ “date”: “2022-02-12T09:04:11+0000”, “dirty”: false, “error”: null, “full-revisionid”: “06d230151e6f18fdb8139d09abf539867a8cd481”, “version”: “1.4.1” }

Issue Analytics

State:
Created a year ago
Comments:6 (6 by maintainers)

Top GitHub Comments

1reaction

rhshadrachcommented, Apr 22, 2022

Thanks for the report!

0reactions

simonjayhawkinscommented, Aug 30, 2022

removing from 1.4.x milestone.

Top Results From Across the Web

strange error in python pandas apply - Stack Overflow

You're getting a key error because of the [0] . While it's not a perfect description, when you specify df.groupby('a').

pandas.DataFrame.apply — pandas 1.5.2 documentation

Apply a function along an axis of the DataFrame. ... Positional arguments to pass to func in addition to the array/series. ... df.apply(lambda...

Different Ways to Change Data Type in pandas

While working in Pandas DataFrame or any table-like data structures we are often ... Pandas generates an error, in order to ignore error...

Spark SQL, DataFrames and Datasets Guide

To use these features, you do not need to have an existing Hive setup. Creating DataFrames. Scala; Java; Python; R. With a SparkSession...

DataFrame.map_partitions - Dask documentation

Apply Python function on each DataFrame partition. ... The partition itself will be the first positional argument, with all other arguments passed after....