TypeError: dtype '<class 'datetime.datetime'>' not understood

df
  valuation_date
0     2018-06-08
1     2018-06-11
2     2018-06-12
3     2018-06-13
4     2018-06-14
5     2018-06-15

type(valuation_dates_df)
Out[16]: pandas.core.frame.DataFrame


type(valuation_dates_df['valuation_date'])
Out[15]: pandas.core.series.Series

To reproduce the error:

dict= {'valuation_date': {0: Timestamp('2018-06-08 00:00:00'),
  1: Timestamp('2018-06-11 00:00:00'),
  2: Timestamp('2018-06-12 00:00:00'),
  3: Timestamp('2018-06-13 00:00:00'),
  4: Timestamp('2018-06-14 00:00:00'),
  5: Timestamp('2018-06-15 00:00:00')}}

pd.DataFrame.from_records(dict)['valuation_date'].astype(datetime.datetime)

valuation_dates_df['valuation_date'].astype(datetime.datetime)
  File "C:\git\mre_x\_tcp\work\win-na-x64-release\py3\mre_venv3\lib\site-packages\pandas\core\dtypes\common.py", line 2029, in pandas_dtype
    raise TypeError("dtype '{}' not understood".format(dtype))
TypeError: dtype '<class 'datetime.datetime'>' not understood

The issue occurs with pandas 0.24.1 and did not occur in 0.23.4.

[matthewgdv]

@jreback

By that logic neither should python int or float be allowed, but they currently are (int seems to default to int32).

So I don’t see why we can’t put practicality over purity here and let users have their shorthand conveniences.

I’d also like to point out that the alternative way to perform this operation:

df['some_col'] = pd.to_datetime(df['some_col'])

also doesn’t specify any units. So I’m not sure that’s a strong argument here.

EDIT:

On re-reading this comment I’m realizing it maybe comes off as a bit combative, particularly since text doesn’t convey tone. I’m sorry if so. I really appreciate what you and the rest of the pandas maintainer team do for so many people. I just figured I’d raise something that has been a repeated annoyance to me in the past, but I respect that not everyone may agree and perhaps with good reason. There might be implementation details I’m unaware of that could make a change like this messier than it at first appears. In terms of the actual reasoning for why I think datetime.datetime should be a legal argument for astype, I hope I’ve made a decent case :p

[abimael-dominguez]

I changed from this:

import pandas as pd
date.astype(datetime)

to this:

import pandas as pd
pd.to_datetime(date)

where ‘date’ is a pandas series of dates in string type before the casting.