Failing to specify numeric columns fails with "could not convert foofoo to numeric"

A minor suggestion to improve error messages: The following code fails, because the meta data does not specify that column A is numeric. However, the error message is somewhat non-intuitive:

import pandas as pd
import dask.dataframe as dd
from dask import delayed

df = pd.DataFrame({"A": [1, 2, 3]})
ddf = dd.from_delayed(delayed(lambda: df), meta=pd.DataFrame(columns=["A"]))
ddf["A"].mean()
# TypeError: Could not convert foofoo to numeric

This is probably a consequence of the data in _simple_fake_mapping in dataframe/utils.py. It would be good to turn the error into something more explicit mentioning the column name + dtype, so that users don’t start to search for foofoo.

[jcrist]

explicit hint that the meta data must be wrong.

If the column is actually an object column, then the metadata doesn’t have to be wrong, the operation is just invalid. I’d rather fix series reductions to check for object dtype, and error explicitly with ValueError("mean reduction not supported for object dtype"). If the metadata was correct, this is a better error message than what pandas throws. If the metadata is wrong, the user will see “object dtype” and should be able to track down their prior mistake.

[bluenote10]

My idea would be to wrap all lines like this into:

with safe_meta_numerics():
    meta = self._meta_nonempty.mean(axis=axis, skipna=skipna)

with an appropriate context manager that catches only TypeError containing “Could not convert” and re-raises with an explicit hint that the meta data must be wrong.

It took me quite some time to make sense of this error, probably because:

• I did explicitly specify meta including dtypes, just with a tiny mistake.
• From a user perspective the .mean() is a lazy operation, so it was surprising to see hints of a numeric operation in the first place.
• The “foofoo” pointed me in the wrong direction, checking my code & data for “foofoo”.