REGR: Behavior change with empty apply in pandas 1.3.0rc1

Problem description

The following (toy) snippet worked with 1.2:

df = pd.DataFrame(columns=["a", "b"])
df["a"] = df.apply(lambda x: x["a"], axis=1)

With 1.3 it fails with ValueError: Columns must be same length as key

Technically this is correct - the apply on an empty frame returns the empty frame so things do not really match.

Expected Output

It still works? Just reporting it here if this is an unintended change. Maybe I missed it, but I did not see this mentioned in the changelog.

The fix is to only call apply when the frame is not empty I guess? I stumbled upon this one when running our test suite.

[simonjayhawkins]

Looking at this issue from purely a backport fix perspective, I doesn’t appear that we have any solutions here (for 1.3.x).

Changing the behavior of apply would not be suitable for a backport.

For backport, we would either need to:

1. revert the change to setitem that caused the regression.
2. catch the apply inconsistency in setitem

I think option 2 has been ruled out https://github.com/pandas-dev/pandas/issues/41997#issuecomment-860671005 and https://github.com/pandas-dev/pandas/issues/41997#issuecomment-860833386

I think option 1 is undesirable (especially late in the 1.3.x branch) since the change was a bugfix and is now correct behavior. https://github.com/pandas-dev/pandas/issues/41997#issuecomment-860629747

I propose we remove this from the 1.3.5 milestone.

[jordantshaw]

I am also experiencing the same issue. Example below.

df = pd.DataFrame(columns=["a", "b"])
df['a'] = df.apply(lambda x: x["a"], axis=1)

I would expect ‘x’ in this case to be an empty Series, but instead it is returning an empty DataFrame. When as Any ideas on expected resolution?