question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

`timestamp` index should be DateTimeIndex, not floats

See original GitHub issue

The idea behind representing fetched prom data (or any data) as a Pandas DataFrame is to enable the processing of such data with all the powerful Pandas features.

But in case of a timeseries a major component of the data is the actual timestamps, which, to be usable, need to be represented as some form of datetime objects and not unix epoch timestamp floats, as they are now.

I propose to consider whether this shouldn’t be changed either by changing the default or at least having all functions give the option of returning the timestamps index column converted to DateTimeIndex (similar to how Pandas’ DataFrame.read_csv() has a parse_dates argument).

The code for this might look like this (though maybe there’s a faster way):

newindex = df.index.to_series().apply(lambda ts: datetime.fromtimestamp(ts))
df.set_index(newindex, inplace=True)

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Reactions:2
  • Comments:5 (1 by maintainers)

github_iconTop GitHub Comments

1reaction
chauhankaranrajcommented, Dec 20, 2021

there might be cases where people don’t care about the timestamps themselves (just need the values) and would like an option of making it go faster by skipping the datetime conversion step.

Yeah that makes sense 👍

However, I’m pretty sure I won’t have the time to look into it before February, so please feel free. 😃

Sounds good, I’ll assign myself to this then 😃

/assign chauhankaranraj

0reactions
chauhankaranrajcommented, Dec 20, 2021

Hey @mmazur, I have added a PR (#234) to change the default dtypes for timestamps in the dataframe classes. Feel free to review it and let us know if you have any feedback!

Read more comments on GitHub >

github_iconTop Results From Across the Web

Convert index floats back to datetime - python - Stack Overflow
The easiest is to to convert dates_ext to Timestamp before assigning it to the index of the new rows:
Read more >
Time series / date functionality — pandas 1.5.2 documentation
pandas supports converting integer or float epoch times to Timestamp and DatetimeIndex . The default unit is nanoseconds, since that is how Timestamp...
Read more >
Pandas: Converting integer or float epoch times to Timestamp ...
Write a Pandas program to convert integer or float epoch times to Timestamp and DatetimeIndex. Sample Solution: Python Code :
Read more >
Part 5 - Working with Time Series Data | ArcGIS API for Python
Passing a series of dates by default returns a DatetimeIndex which can be used to index data in a Series or DataFrame ....
Read more >
DateTime in Pandas: A Simple Guide for Beginners (2022)
If you're not familiar with the pandas library, you might like to try our ... If you pass a single integer or float...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found