to_sql method turns datetime64 index to time zone aware in postgres
See original GitHub issueI have a pandas
Dataframe
with an index which is of type datetime64[ns]
.
when I use the to_sql
method i expect the index to be created as a timestamp
postgres column, however it is creates a timestamptz
column.
Note that when the datetime64[ns]
column is not an index but rather a normal column, this doesn’t happen.
pandas 0.23.4, psycopg2 2.7.4, sqlalchemy 1.2.7, PostgreSQL 9.6.6
example
dates = pd.date_range('2018-01-01', periods=5, freq='6h')
df_test = pd.DataFrame({'nums': range(5)}, index=dates)
nums | |
---|---|
2018-01-01 00:00:00 | 0 |
2018-01-01 06:00:00 | 1 |
2018-01-01 12:00:00 | 2 |
inserting to postgres
df_test.to_sql('foo_table',postgres_uri,schema='data_test',index_label='info_date')
when reading from the database I get the index with different type
df_db = pd.read_sql_table('foo_table',postgres_uri,schema='data_test',index_col='info_date')
nums | |
---|---|
info_date | |
2018-01-01 00:00:00+00:00 | 0 |
2018-01-01 06:00:00+00:00 | 1 |
2018-01-01 12:00:00+00:00 | 2 |
not sure if it’s a sqlalchemy or pandas question. reported because of my question on SO
Issue Analytics
- State:
- Created 5 years ago
- Comments:5 (4 by maintainers)
Top Results From Across the Web
pandas to sql turns datetime64 index to time zone aware ...
[Solved]-pandas to sql turns datetime64 index to time zone aware-postgresql. Search. score:0. I use mysql , but maybe if you set index=True?
Read more >pandas to sql turns datetime64 index to time zone aware
when I use the to_sql method i expect the index to be created as a timestamp postgres column, however it is creates a...
Read more >Documentation: 15: 8.5. Date/Time Types - PostgreSQL
PostgreSQL supports the full set of SQL date and time types, shown in Table ... In most cases, a combination of date ,...
Read more >14 Time Zone Handling — Pandas Doc - GitHub Pages
To remove timezone from tz-aware DatetimeIndex , use tz_localize(None) or tz_convert(None) . tz_localize(None) will remove timezone holding local time ...
Read more >Exploring Postgres date formats and their different functions
Date format: This is where you specify the new date format. The following query converts existing date values stored in the [SalesOrders] table ......
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
I was able to confirm this behavior on master. I’ve included a fix in #22654 since it was a simple fix to handle.
@mroeschke it’s a normal schema, I just didn’t want to create a table in my
public
schema.