Weekly scheduler not firing current week execution

Dear Airflow Maintainers,

Before I tell you about my issue, let me describe my environment:

Environment

• Version of Airflow: 1.7.0
• Airflow components and configuration: Airflow running a SequentialExecutor.
• DAG code

from airflow import DAG
from airflow.operators import BashOperator
from datetime import datetime, timedelta

from airflow.hooks import DynamoDBHook

default_args = {
    'owner': 'airflow',
    'depends_on_past': True,
    'start_date': datetime(2016, 3, 15, 8, 0),
    'email': ['[e-mail]'],
    'email_on_failure': False,
    'email_on_retry': False,
    'retries': 1,
    'retry_delay': timedelta(minutes=5)
}

dag = DAG('weekly_dag', default_args=default_args, schedule_interval='0 8 * * 2')

t1 = BashOperator(task_id='bc', bash_command='echo {{ts}} >> ~/test.txt', dag=dag)

• Screen shots of your DAG’s graph and tree views:
• Operating System: Linux sdiazb 4.2.0-35-generic #40-Ubuntu SMP Tue Mar 15 22:15:45 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
• Python Version: Python 2.7.10

Now that you know a little about me, let me tell you about the issue I am having:

Description of Issue

Hi all,

I am trying to run a dummy weekly dag with one task that should be triggered every tuesday since 2016-03-15. However, when I am running the scheduler today (2016-04-13), the execution for the current week (2016-04-12) has not been scheduled. Am I missing something?

• What did you expect to happen? Five runs should have been scheduled: 2016-03-15, 2016-03-22, 2016-03-29, 2016-04-05, 2016-04-12.
• What happened instead? When running airflow scheduler on 2016-04-13, the execution for tuesday (2016-04-12) run wasn’t fired (image attached)
• Here is how you can reproduce this issue on your machine:

Reproduction Steps

1. Import dag to dags folder
2. Run webserver airflow webserver -p 8080
3. Run scheduler airflow scheduler

[jlowin]

@sdiazb this is a very common pitfall. Execution dates refer to the start of a period; Airflow won’t actually run the dag until the period is over. It’s easier to see with a daily dag: the 4/12/2016 run won’t commence until just after midnight on 4/13/2016. In other words, when the day represented by 4/12/2016 is completely past.

In your case, we are still in the weekly interval starting on 4/12/2016 and lasting until 4/19/16. I would expect that on 4/19/16, your DAG will fire with the 4/12/16 execution date.

This can be very confusing with intervals of longer than 1 day because the “execution_date” is not actually the day it gets executed… however it is a very difficult thing to change at this stage.

[sdiazb]

Thanks @mistercrunch. It was my fault, I was confused when testing _backfill _and _scheduler _and I didn’t interpret the documentation in the right way.

Anyway I still have a doubt. Is there a workaround so you can trigger a weekly DAG and not being one week before the scheule? I am pretty sure that this could be implemented with daily schedule and branch operators but I wonder if there is a cleaner solution.

Thank you very much all of you for your help and for making Airflow such a great tool!