Task Instances stuck in "scheduled" state
See original GitHub issueApache Airflow version: 2.0.1
Kubernetes version (if you are using kubernetes) (use kubectl version
): v1.17.12
Environment:
- Cloud provider or hardware configuration: AWS EKS
- OS (e.g. from /etc/os-release):
- Kernel (e.g.
uname -a
): - Install tools:
- Others:
What happened:
Several Task Instances get scheduled
but never move to queued
or running
. They then become orphan tasks.
What you expected to happen: Tasks to get scheduled and run 😃
How to reproduce it:
Anything else we need to know: I believe the issue is caused by this limit. If we have more Task Instances than Pool Slots Free then some Task Instances may never show up in this query.
Issue Analytics
- State:
- Created 3 years ago
- Comments:8 (7 by maintainers)
Top Results From Across the Web
Task stuck in "scheduled" or "queued" state, pool has all slots ...
Tasks in "queued" state say Task is in the 'running' state which is not a valid state for execution. The task must be...
Read more >Airflow tasks get stuck at “Scheduled” status and never gets ...
Tasks stuck in a “queued” state usually mean one of two things, no queue to execute on or no pool to execute in....
Read more >Tasks are staying in scheduled forever and not running, and ...
Task instance info for each of the stuck tasks (all of the tasks) lists this as the reason: All dependencies are met but...
Read more >7 Common Errors to Check When Debugging Airflow DAGs
7 Common Errors to Check When Debugging Airflow DAGs. Tasks not running? DAG stuck? Logs nowhere to be found? We've been there.
Read more >Why my Airflow tasks got stuck in “no_status” and how I fixed it
One of our Airflow DAGs were not scheduling tasks. ... and running half of the tasks, but the other half got stuck in...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
Yep, we’re using the KubernetesExecutor
Thanks @jedcunningham – assigned to you