Dag dependency view is not rendering for Postgres backed Airflow

Apache Airflow version

2.2.2

What happened

Dag dependency view is not rendering for Postgres backed Airflow.

What you expected to happen

In this if statements, the iterator for the else part (which is including Postgres) is assigned as session.query object which is shouldn’t be. Current :

if session.bind.dialect.name in ["sqlite", "mysql"]:
    query = session.query(cls.dag_id, func.json_extract(cls.data, "$.dag.dag_dependencies"))
    iterator = ((dag_id, json.loads(deps_data) if deps_data else []) for dag_id, deps_data in query)
elif session.bind.dialect.name == "mssql":
    query = session.query(cls.dag_id, func.json_query(cls.data, "$.dag.dag_dependencies"))
    iterator = ((dag_id, json.loads(deps_data) if deps_data else []) for dag_id, deps_data in query)
else:
    iterator = session.query(cls.dag_id, func.json_extract_path(cls.data, "dag", "dag_dependencies"))
return {dag_id: [DagDependency(**d) for d in (deps_data or [])] for dag_id, deps_data in iterator}

It should be like this :

if session.bind.dialect.name in ["sqlite", "mysql"]:
    query = session.query(cls.dag_id, func.json_extract(cls.data, "$.dag.dag_dependencies"))
elif session.bind.dialect.name == "mssql":
    query = session.query(cls.dag_id, func.json_query(cls.data, "$.dag.dag_dependencies"))
else:
    query = session.query(cls.dag_id, func.json_extract_path(cls.data, "dag", "dag_dependencies"))

iterator = ((dag_id, json.loads(deps_data) if deps_data else []) for dag_id, deps_data in query)
return {dag_id: [DagDependency(**d) for d in (deps_data or [])] for dag_id, deps_data in iterator}

How to reproduce

1. Deploy an Airflow instance with Postgres as the meatadata DB
2. Create two dags (one dag depends to the other), it can use TriggerDagRunOperator or ExternalTaskSensor
3. Check the Dag dependencies view from the UI : Browse ->. Dag dependencies

Operating System

Debian GNU/Linux 10 VERSION_ID=“10”

Versions of Apache Airflow Providers

apache-airflow 2.2.2 apache-airflow-providers-amazon 2.4.0 apache-airflow-providers-celery 2.1.0 apache-airflow-providers-cncf-kubernetes 2.1.0 apache-airflow-providers-databricks 2.0.2 apache-airflow-providers-docker 2.3.0 apache-airflow-providers-elasticsearch 2.1.0 apache-airflow-providers-ftp 2.0.1 apache-airflow-providers-google 6.1.0 apache-airflow-providers-grpc 2.0.1 apache-airflow-providers-hashicorp 2.1.1 apache-airflow-providers-http 2.0.1 apache-airflow-providers-imap 2.0.1 apache-airflow-providers-jira 2.0.1 apache-airflow-providers-microsoft-azure 3.3.0 apache-airflow-providers-microsoft-mssql 2.0.1 apache-airflow-providers-mongo 2.2.0 apache-airflow-providers-mysql 2.1.1 apache-airflow-providers-odbc 2.0.1 apache-airflow-providers-opsgenie 2.0.1 apache-airflow-providers-postgres 2.3.0 apache-airflow-providers-redis 2.0.1 apache-airflow-providers-salesforce 3.3.0 apache-airflow-providers-sendgrid 2.0.1 apache-airflow-providers-sftp 2.2.0 apache-airflow-providers-slack 4.1.0 apache-airflow-providers-sqlite 2.0.1 apache-airflow-providers-ssh 2.3.0 apache-airflow-providers-tableau 2.1.2

Deployment

Docker-Compose

Deployment details

No response

Anything else

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project’s Code of Conduct

[potiuk]

Hmmm maybe it would be a good idea to add it as PR to https://airflow.apache.org/docs/apache-airflow/stable/howto/operator/external_task_sensor.html?highlight=dag dependency (I guess this is the right page)

I think the two f you @lidalei @zambadruzaman are one of the two best people out there to write such a description tha will be helpful and useful for people like you who will struggle with it 😃. This is really a good way to give back in return for the free software you get ! You can become one of the more thab 1900 (and approaching 2000 fast) number of contributors to Airflow?

This is very simple. Just click “Suggest chnage on this page” at the bottom right and you will get a PR where you will be able to add the change. What say you ?

[zambadruzaman]

Hi @lidalei, yes it was the exact same issue, when you delete any dags, make sure there is no other dag depends on the deleted dag. If you have a dag which has external trigger or sensor pointing to the deleted dag, the dependency graph will be broken and the dagre-d3 js won’t able to render the graph.