date time object time zone information not preserved in output

Description of problem:

Observed behavior: Depending on the chosen output format, psort outputs inconsistent datetime values with regard to time shifts and timezone information.

Expected behavior: Timezone information is consistent between the different output formats.

Example: For the xlsx format, the following timeline data is generated (these are the correct timestamps in UTC, the source data is in UTC+2):

datetime			timestamp_desc		source	source_long
2021-06-30 09:39:01,000	Content Modification Time	LOG	Cron log
2021-06-30 09:39:01,000	Content Modification Time	LOG	Log File
2021-06-30 09:39:01,000	Content Modification Time	LOG	Log File
2021-06-30 09:39:01,000	Content Modification Time	LOG	Log File
2021-06-30 09:39:01,000	Content Modification Time	LOG	Log File
2021-06-30 09:39:01,000	Content Modification Time	LOG	Log File
2021-06-30 09:39:01,000	Content Modification Time	LOG	Log File
2021-06-30 09:39:01,000	Content Modification Time	LOG	Log File
2021-06-30 09:39:01,000	Content Modification Time	LOG	Log File
2021-06-30 10:00:18,000	Event Recorded		LOG	Apache Access
2021-06-30 10:00:18,000	Event Recorded		LOG	Apache Access
2021-06-30 10:00:19,000	Event Recorded		LOG	Apache Access
2021-06-30 10:00:19,000	Event Recorded		LOG	Apache Access
2021-06-30 10:00:19,000	Event Recorded		LOG	Apache Access
2021-06-30 10:00:19,000	Event Recorded		LOG	Apache Access
2021-06-30 10:00:19,000	Event Recorded		LOG	Apache Access
2021-06-30 10:00:19,000	Event Recorded		LOG	Apache Access
2021-06-30 10:00:19,000	Event Recorded		LOG	Apache Access
2021-06-30 10:00:19,000	Event Recorded		LOG	Apache Access
2021-06-30 10:01:06,000	Content Modification Time	LOG	Log File
2021-06-30 10:01:09,000	Content Modification Time	LOG	Log File
[...]

The same timeline, but with the dynamic format (and otherwise unchanged parameters):

datetime			timestamp_desc			source	source_long
2021-06-30T09:39:01+00:00	Content Modification Time	LOG	Cron log
2021-06-30T11:39:01+00:00	Content Modification Time	LOG	Log File
2021-06-30T11:39:01+00:00	Content Modification Time	LOG	Log File
2021-06-30T11:39:01+00:00	Content Modification Time	LOG	Log File
2021-06-30T11:39:01+00:00	Content Modification Time	LOG	Log File
2021-06-30T11:39:01+00:00	Content Modification Time	LOG	Log File
2021-06-30T11:39:01+00:00	Content Modification Time	LOG	Log File
2021-06-30T11:39:01+00:00	Content Modification Time	LOG	Log File
2021-06-30T11:39:01+00:00	Content Modification Time	LOG	Log File
2021-06-30T12:00:18+00:00	Event Recorded		LOG	Apache Access
2021-06-30T12:00:18+00:00	Event Recorded		LOG	Apache Access
2021-06-30T12:00:19+00:00	Event Recorded		LOG	Apache Access
2021-06-30T12:00:19+00:00	Event Recorded		LOG	Apache Access
2021-06-30T12:00:19+00:00	Event Recorded		LOG	Apache Access
2021-06-30T12:00:19+00:00	Event Recorded		LOG	Apache Access
2021-06-30T12:00:19+00:00	Event Recorded		LOG	Apache Access
2021-06-30T12:00:19+00:00	Event Recorded		LOG	Apache Access
2021-06-30T12:00:19+00:00	Event Recorded		LOG	Apache Access
2021-06-30T12:00:19+00:00	Event Recorded		LOG	Apache Access
2021-06-30T12:01:06+00:00	Content Modification Time	LOG	Log File
2021-06-30T12:01:09+00:00	Content Modification Time	LOG	Log File
[...]

Note that the timestamps in the dynamic output are all over the place regarding time shifts and the given timezones are incorrent. Nonetheless, the correct order of the entries is preserverd. Something similar can be observed for l2tcsv output.

Command line and arguments:

Command to generate the plaso file from source data:

docker run -v "$PWD:/data" --rm -it log2timeline/plaso log2timeline --parsers=linux,apache_access --file-filter=/data/filter.txt --partitions 1 --storage_file /data/server.plaso /data/server.E01

Command to generate the xlsx output:

docker run -v "$PWD:/data" --rm -it log2timeline/plaso psort -o xlsx -w /data/server.xlsx /data/server.plaso 'date > DATETIME("2021-06-30T11:30:00+02:00")' 

docker run -v "$PWD:/data" --rm -it log2timeline/plaso psort -o dynamic -w /data/server.csv /data/server.plaso 'date > DATETIME("2021-06-30T11:30:00+02:00")'

Source data:

I can provide this to anyone interested. Just an E01 of a fairly minimal Debian 10 instatllation with LAMP stack and Wordpress for testing purposes.

Plaso version:

Current version from Docker Hub:

$ docker run --rm -it log2timeline/plaso log2timeline --version
plaso - log2timeline version 20210606

Operating system Plaso is running on:

Whatever the container uses internally, Docker host is Debian Sid.

Installation method:

Docker.

Debug output/tracebacks:

Nothing meaningful in both debug and log output of psort.

[joachimmetz]

thx having a closer look shortly

[joachimmetz]

changes merged closing issue.