question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Paused DAG manual running task has no external_ executor_ id

See original GitHub issue

Apache Airflow version

2.1.4

What happened

When we have a simple DAG and execute the sleep command to keep running, if we pause the DAG first and manually clear a task, it will not run at this time. Now let’s run the task directly manually. The status of the task is running and the status of the whole DAG is queued. Now, the operation that causes the bug comes. Unpause the dag and the scheduler log will report

2022-04-06 11:39:34,250] {scheduler_job.py:1283} INFO - Reset the following 1 orphaned TaskInstances:

<TaskInstance: dag. test 2022-04-04 16:00:00+00:00 [running]>

Found field external_executor_id in table task_instance is empty, resulting in adopt_ or_ reset_ orphaned_ Tasks cannot be used correctly

The above case is just a simple reproduction of the problem. This problem occasionally occurs during the operation of our complex formal tasks. Usually, the error is reported as follows

[2022-04-02 19:22:19,365] {local_task_job.py:209} WARNING - 
State of this instance has been externally set to None. Terminating instance.

What you think should happen instead

Running tasks are not reset

How to reproduce

No response

Operating System

celery executor | k8s

Versions of Apache Airflow Providers

No response

Deployment

Other

Deployment details

No response

Anything else

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Issue Analytics

  • State:closed
  • Created a year ago
  • Comments:5 (2 by maintainers)

github_iconTop GitHub Comments

2reactions
wangzhijunwangcommented, Apr 13, 2022

There are no current problems in version 2.2.4

0reactions
tanelkcommented, Apr 12, 2022

Did I understand correctly - in 2.2.t the external_executor_id is filled, but the issue still there?

Read more comments on GitHub >

github_iconTop Results From Across the Web

Airflow 1.9.0 is queuing but not launching tasks
The actual problem is that the dag is in the pause state. On toggling the dag state the tasks would start to execute....
Read more >
airflow.models.dag — Airflow Documentation
Creates a dag run from this dag including the tasks associated with this dag. Returns the dag run. Parameters. run_id (str | None)...
Read more >
7 Common Errors to Check When Debugging Airflow DAGs
Tasks not running ? DAG stuck? Logs nowhere to be found? We've been there. Here's a list of common snags and some corresponding...
Read more >
Scheduling & Triggers - Apache Airflow Documentation
The Airflow scheduler monitors all tasks and all DAGs, and triggers the task ... Each DAG may or may not have a schedule,...
Read more >
How to Stop or Kill Airflow Tasks: 2 Easy Methods
If you stop a DAG and clear the task from the UI, the running tasks in the executor will not stop. When the...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found