Improve datetime feature handling
See original GitHub issueRelated: #1433
In AutoGluon when datetime features are detected, they are simply converted to an int.
Instead, we could generate a variety of additional features, such as:
- Year, Month, Day (#1446)
- Day of Week (#1446)
- Is Holiday / Near Holiday
Additionally, we could improve the detection of date features:
- Detect datetime in datetime_as_int form (such as
1533140820000000000
, equivalent to8/1/2018 16:27
) - Detect datetime in yyyymmdd int/str form (such as
20211222
)
If you are interested in contributing this functionality, please respond to this issue.
Current datetime feature generator: https://github.com/awslabs/autogluon/blob/master/features/src/autogluon/features/generators/datetime.py
Current datetime unit test: https://github.com/awslabs/autogluon/blob/master/features/tests/features/generators/test_datetime.py
Issue Analytics
- State:
- Created 2 years ago
- Comments:6 (3 by maintainers)
Top Results From Across the Web
Improve datetime feature handling · Issue #1436 · autogluon ...
In AutoGluon when datetime features are detected, they are simply converted to an int. Instead, we could generate a variety of additional ...
Read more >Machine Learning with Datetime Feature Engineering
In this post, I will demonstrate how to create datetime features with built in pandas functions for your machine learning models.
Read more >Dealing with DateTime Features in Python and Pandas
Being able to master these DateTime features will help you go a long way towards becoming a better (and more efficient) data scientist....
Read more >Handle the Datetime and coordinates Features
In the real-life dataset, Datetime and coordinates features are often present. To get useful information from Datetime and Coordinates features are also a ......
Read more >Feature Engineering of DateTime Variables - Kaggle
Feature Engineering of DateTime Variables · 1: Extract Date time Component · 2: Create Day Part Flag · 3 : Create Boolean Flags...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
I have a pull request + new tests ready to go. It adds ‘year’, ‘month’, ‘day’, ‘hour’, ‘minute’, ‘second’, ‘dayofweek’, ‘weekday’, ‘dayofyear’, ‘quarter’, ‘is_month_end’, ‘is_leap_year’ I think all the pd.Series.dt.* can be derived from these. I didn’t use FastAI to avoid an extra dependency.
https://github.com/awslabs/autogluon/pull/1444
Thanks to @willsmithorg for contributing #1446 which adds improved datetime handling logic to AutoGluon!
@trujillo9616 if you are still interested in working on this logic, please refer to #1446 for the current state of the code, as the main logic is now implemented. An additional datetime feature that isn’t yet implemented is
is_holiday
.