Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Return timestamp values as datetime objects

See original GitHub issue

Several Scrapinghub API endpoints accept or return timestamps, currently as UNIX timestamp in milliseconds.

It would be great to have those values as datetime.datetime objects in the results so that consumers of python-scrapinghub calls do not have to convert them for interpretation.

Passing datetime.datetime objects in methods allowing filtering on timestamps, e.g. where startts and endts arguments are supported, would be very handy too.

Issue Analytics

State:
Created 6 years ago
Comments:8 (8 by maintainers)

Top GitHub Comments

1reaction

chekunkovcommented, Apr 25, 2017

I do see “finished_time”, “running_time”, “pending_time” and “ts” in many of your examples.

I forgot add a comment on this. Yes, technically we can recursively check response for the given fields + check request url for endpoints like /jobs/108106/13/2879/pending_time, but as I mentioned before my main concerns are performance (maybe not significant, we will measure it for sure if we select this for development), lack of support for new timestamp fields if we add some in the old clients, potential false positive conversions when client will convert user defined fields with the same names - it’s possible to have such field names in spider arguments, items, collections - to avoid that we will need to define a set of endpoints or fields inside the responses that shouldn’t be converted. That brings the complexity I was talking about and that’s why I don’t like this idea.

0reactions

redapplecommented, Apr 25, 2017

Alright, I’ll keep this open as nice to have then.

Top Results From Across the Web

Python timestamp to datetime and vice-versa - Programiz

You can get timestamp from a datetime object using datetime.timestamp() method. from datetime import datetime # current date and time now = ...

How to convert timestamp string to datetime object in Python?

Timestamp to DateTime object. You can simply use the fromtimestamp function from the DateTime module to get a date from a UNIX timestamp....

How to convert a Timestamp object in pandas to Python's ...

Return value. The to_pydatetime() function returns a datetime object that returns the same date and time value from the input Timestamp object.

Why does pandas return timestamps instead of datetime ...

1 Answer 1 · pandas stores datetimes as data with type datetime64 in index/columns (this are not datetime.datetime objects). · when retrieving one ......

How to convert timestamp string to datetime ... - Tutorialspoint

Use the type() function(returns the data type of an object) to print the type of resultant datetime object. Example. The following program ...