DOC: Dataframe.apply does not always return a Series when reduce=True

Code Sample, a copy-pastable example if possible

pd.DataFrame({'col_1':['val_1','val_2','val_3']}).apply(lambda row: [1], axis=1, reduce=True)

Problem description

Per the documentation, calling Dataframe.apply w/ reduce=True should always produce a Series. However, if the applied function returns list values Dataframe.apply returns a Dataframe, not a Series.

Expected Output

When called with reduce=True, I expect Dataframe.apply to produce a Series where each element is a list, not a Dataframe.

Output of pd.show_versions()

0.18.1

[jorisvandenbossche]

@jreback I agree that apply does a lot of guessing in the default case, and it is sometimes difficult to predict what the output will be. But isn’t that the point of the keyword? To be able to specify this and be sure about the resulting shape? In any case, if this is a won’t fix, then the documentation is wrong.

[jreback]

reduce : boolean or None, default None
    Try to apply reduction procedures. If the DataFrame is empty,
    apply will use reduce to determine whether the result should be a
    Series or a DataFrame. If reduce is None (the default), apply's
    return value will be guessed by calling func an empty Series (note:
    while guessing, exceptions raised by func will be ignored). If
    reduce is True a Series will always be returned, and if False a
    DataFrame will always be returned.

I guess the doc-string is misleading. reduce really only applies to empty frames and has no effect otherwise, which it says in the end.

I suppose some nice examples of what not to do in .apply (like returning a 1-element list and expecting it work, rather returning tuples) might be in order.

@NelsonAndrew I’ll re-open as a doc issue.

.apply is already pretty magical, I think it could use a comprehensive fix but that is likely to break back-compat.