New grid view in Airflow 2.3.0 has very slow performance on large DAGs relative to tree view in 2.2.5

Apache Airflow version

2.3.0 (latest released)

What happened

I upgraded a local dev deployment of Airflow from 2.2.5 to 2.3.0, then loaded the new /dags/<dag_id>/grid page for a few dag ids.

On a big DAG, I’m seeing 30+ second latency on the /grid API, followed by a 10+ second delay each time I click a green rectangle. For a smaller DAG I tried, the page was pretty snappy.

I went back to 2.2.5 and loaded the tree view for comparison, and saw that the /tree/ endpoint on the large DAG had 9 seconds of latency, and clicking a green rectangle had instant responsiveness.

This is slow enough that it would be a blocker for my team to upgrade.

What you think should happen instead

The grid view should be equally performant to the tree view it replaces

How to reproduce

Generate a large DAG. Mine looks like the following:

• 900 tasks
• 150 task groups
• 25 historical runs

Compare against a small DAG, in my case:

• 200 tasks
• 36 task groups
• 25 historical runs

The large DAG is unusable, the small DAG is usable.

Operating System

Ubuntu 20.04.3 LTS (Focal Fossa)

Versions of Apache Airflow Providers

No response

Deployment

Docker-Compose

Deployment details

Docker-compose deployment on an EC2 instance running ubuntu. Airflow web server is nearly stock image from apache/airflow:2.3.0-python3.9

Anything else

Screenshot of load time:

GIF of click latency:

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project’s Code of Conduct

[bbovenzi]

Yes, that change was just for dynamic tasks. I am working on more optimizations. They just didn’t make it in time for 2.3.1.

[bbovenzi]

Going to reopen as we can still do more to improve perfomance for large DAGs