Fix naive-datetime timezone conversion when marshaling dataframes

Getting a NonExistentTimeError when marshalling naive datetimes in Pandas dataframes. Specifically when marshaling 2020-8-3 2:30 from a user in PST timezone.

Related to discussion

This is because in the US, daylight savings time starts at 2 am and clocks are turned forward one hour to 3 am, so 2:30 doesn’t exist on March 8. This is fine. However, I never said my datetime was in a US timezone, or any timezone at all. Streamlit localized my datetime to some timezone (presumably ‘America/Los_Angeles’, as I’m in San Francisco).

The code in question is in the _marshall_any_array function in elements/data_frame_proto.py. There’s even a note that says # TODO(armando): Convert eveything to UTC not local timezone. Converting to UTC is an improvement, but still not as good as just leaving it alone and displaying it as is.

[skovhus]

An alternative might be setting the timezone environment variable:

import os

os.environ['TZ'] = 'UTC' # Hardcoded as workaround to  https://github.com/streamlit/streamlit/issues/1061

[karriebear]

@crypticintent’s solution is definitely the right approach. Any datetimes that have a timezone will be left as is. Thanks for sharing that!

Unfortunately our current implementation of timezone awareness is not correct but it is something we are actively looking into. Forcing timezone awareness helps us ensure datetimes display in sync between the browser and server side, especially when they are in different timezones.

We’re hoping to have a solution out that will support as many use cases as possible. In the meantime, we’ll take a look at expediting a fix to localize to UTC if no timezone exists.