Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

BUG: Index.astype does not localize to UTC first like Series.astype with datetime64[ns, tz] dtypes

See original GitHub issue

I have checked that this issue has not already been reported.
I have confirmed this bug exists on the latest version of pandas.
(optional) I have confirmed this bug exists on the master branch of pandas.

Code Sample, a copy-pastable example

In [4]: pd.__version__
Out[4]: '1.1.0.dev0+1202.g1de2cf1ac'

In [5]: pd.Series([pd.Timestamp('2020-01-01 15:00')]).astype('datetime64[ns, EST]')
Out[5]:
0   2020-01-01 10:00:00-05:00
dtype: datetime64[ns, EST]

In [6]: pd.Index([pd.Timestamp('2020-01-01 15:00')]).astype('datetime64[ns, EST]')
Out[6]: DatetimeIndex(['2020-01-01 15:00:00-05:00'], dtype='datetime64[ns, EST]', freq=None)

Problem description

Out[5] is the expected output as documented in https://pandas.pydata.org/docs/user_guide/timeseries.html#time-zone-series-operations. Out[6] should align therefore with Out[5]

Expected Output

In [6]: pd.Index([pd.Timestamp('2020-01-01 15:00')]).astype('datetime64[ns, EST]')
Out[6]: DatetimeIndex(['2020-01-01 10:00:00-05:00'], dtype='datetime64[ns, EST]', freq=None)

Issue Analytics

State:
Created 3 years ago
Comments:14 (14 by maintainers)

Top GitHub Comments

1reaction

jbrockmendelcommented, Oct 13, 2020

I think the Technically Correct answer may be to raise and require users to use tz_localize

The semantics of astype usually mean “represent the same information in another dtype”. When going from tznaive<->tzaware, no matter which convention we use, the result does not represent “the same information”

0reactions

mroeschkecommented, Jun 11, 2021

This was deprecated in https://github.com/pandas-dev/pandas/pull/39258 which makes this bug a nonissue. Closing

Top Results From Across the Web

What's new in 1.5.0 (September 19, 2022) - Pandas

In a future version of pandas, not specifying group_keys will default to the same ... DatetimeIndex.astype() now supports casting timezone-naive indexes to ...

convert datetime64[ns, UTC] pandas column to datetime

To remove timezone, use tz_localize : df['timestamp'] = pd.to_datetime(df.timestamp).dt.tz_localize(None). Output:

What's new in 0.24.0 (January 25, 2019) - Pandas 中文

The 0.24.x series of releases will be the last to support Python 2. ... other dtypes In [6]: s + s.iloc[1:3].astype('Int8') Out[6]: 0...

dask.dataframe.to_datetime - Dask documentation

This function converts a scalar, array-like, Series or DataFrame /dict-like ... To do this, timezone-naive inputs are localized as UTC, ... DataFrame.astype.

What's New — pandas 0.23.4 documentation

Now, to_datetime() will correctly localize Series with a datetime64[ns, UTC] dtype to be consistent with how list-like and Index data are handled. (GH6415)....