max_active_runs = 1 can still create multiple active execution runs

Edit: There is a separate issue affecting max_active_runs in 1.10.14. That regression is fixed in 1.10.15.

Edit2: Version v2.1.3 contains some fixes but also contains bad regressions involving max_active_runs. Use v2.14 for the complete fixes to this issue

Edit3: Version 2.2.0 contains a fix for max_active_runs using dags trigger command or TriggerDagRunOperator. https://github.com/apache/airflow/issues/18583

–

Apache Airflow version: 1.10.11, localExecutor

What happened:

I have max_active_runs = 1 in my dag file (which consists of multiple tasks) and I manually triggered a dag. While it was running, a second execution began under its scheduled time while the first execution was running.

I should note that the second execution is initially queued. It’s only when the dag’s 1st execution moves to the next task that the second execution actually starts.

My dag definition. The dag just contains tasks using pythonOperator.

dag = DAG(
    'dag1',
    default_args=default_args,
    description='xyz',
    schedule_interval=timedelta(hours=1),
    catchup=False,
    max_active_runs=1
)

What you expected to happen:

Only one execution should run. A second execution should be queued but not begin executing.

How to reproduce it: In my scenario:

1. Manually trigger dag with multiple tasks… have task1 take longer than the beginning of the next scheduled execution. (Dag Execution1). As an example, if the scheduled interval is 1 hour, have task1 take longer than 1 hour so as to queue up the second execution (Execution2).
2. When task1 of Execution1 finishes and just before starting task2, the second execution (Execution2, which is already queued) begins running.

Anything else we need to know: I think the second execution begins in between the task1 and task2 of execution1. I think there’s a few second delay there and maybe that’s when Airflow thinks there’s no dag execution? That’s just a guess.

Btw, this can have potentially disastrous effects (errors, incomplete data without errors, etc)

[mik-laj]

The problem is, we don’t have a state that describes DAG Run that are saved but not running. All DAG Run have running state initially. If we want to fix this bug we have to add a new dag state.

[fj-sanchez]

Yeah, this is actually important for us also, it would be great to get this fixed ASAP. Also, currently it’s the bug with the biggest amount of 👍