question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

sla_miss.execution_date is incorrect for daily dagruns

See original GitHub issue

Apache Airflow version

2.2.4 (latest released)

What happened

When inspecting metadata in airflow tables about sla stuff found that

  • in case of daily DAG’s schedule execution_date attribute for sla_miss is filled with unexpected value. Seems like data_interval_end image
  • in case of hourly schedule sla_miss.execution_date are as expected (equals to DAGRun’s data_interval_start/logical_date/execution_date) image image

this prevents to create any custom SLA analisys dashboards/metrics because unable to join sla_miss and dag_run/task_instance in direct and clear way

clicking on daily task’s link on SLA Miss dashboard forwarding to task instance which was never executed (because it is in “future”) and have no runtime metrics in Task instance details

All screenshots are taken @ 2022-03-25 17:xx:xx Z image

What you think should happen instead

sla_miss.execution_date should be same as dag_run.execution_date in all cases

How to reproduce

setup a dag with any daily schedule setup an SLA on task wait sla miss to trigger observe in SLA Miss dashboard - incorrect incorrect execution date

Operating System

amazon linux

Deployment

Official Apache Airflow Helm Chart

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Issue Analytics

  • State:open
  • Created a year ago
  • Comments:8 (7 by maintainers)

github_iconTop GitHub Comments

1reaction
uranusjrcommented, Apr 1, 2022

We should advertise 2.4 as “SLA now actually works!”

0reactions
potiukcommented, Jul 18, 2022

Actully - If you are much into it and REALLY want to dig into it and see if you can incrementally improve it - I think that might be a super-valuable contribution to the community 😃. So if you have the will, I am happy to help with reviews and comments 😃. Just ping me in the PRs.

Read more comments on GitHub >

github_iconTop Results From Across the Web

airflow.models.dag — Airflow Documentation
This calculates what time interval the next DagRun should operate on (its execution date) and when it can be scheduled, according to the...
Read more >
Unable to run Airflow Tasks due to execution date and start date
So, when you schedule your dag, any dag_run's execution_date will be smaller than its start time. For daily, there will be 24 hours...
Read more >
API Reference - Apache Airflow Documentation - Read the Docs
In Airflow, the daily task stamped with execution_date 2016-01-01 can only start ... execution_date – The execution date of the DagRun to find....
Read more >
Data Pipelines with Apache Airflow - BI Consult
the main interface for users to monitor DAG runs and their results. Pipeline as DAG. Task 1. Task 2. Task 3. Task 4....
Read more >
API Reference — Airflow Documentation
If you are looking to delay a task, for example running a daily task at 2AM, ... :param execution_date: The execution date of...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found