Airflow upgrade_check gives false positives on DbApi functions implementations check
See original GitHub issueApache Airflow version: 1.10.14
Kubernetes version (if you are using kubernetes) (use kubectl version
):
N/A
Environment:
- Cloud provider or hardware configuration: GCP
- OS (e.g. from /etc/os-release): Ubuntu 18.04.4 LTS
- Kernel (e.g.
uname -a
): N/A - Install tools: pip
What happened:
Running airflow upgrade_check
threw the following warnings:
1. Class <class 'airflow.hooks.http_hook.HttpHook'> incorrectly implements the function run while inheriting from BaseHook. Please make this class inherit from airflow.hooks.db_api_hook.DbApiHook instead
2. Class <class 'airflow.contrib.hooks.bigquery_hook.BigQueryHook'> incorrectly implements the function get_pandas_df while inheriting from BaseHook. Please make this class inherit from airflow.hooks.db_api_hook.DbApiHook instead
3. Class <class 'airflow.contrib.hooks.bigquery_hook.BigQueryHook'> incorrectly implements the function run while inheriting from BaseHook. Please make this class inherit from airflow.hooks.db_api_hook.DbApiHook instead
4. Class <class 'airflow.contrib.hooks.bigquery_hook.BigQueryHook'> incorrectly implements the function get_records while inheriting from BaseHook. Please make this class inherit from airflow.hooks.db_api_hook.DbApiHook instead
5. Class <class '_hooks.gc_bigquery_hook.BigQueryStandardHook'> incorrectly implements the function get_pandas_df while inheriting from BaseHook. Please make this class inherit from airflow.hooks.db_api_hook.DbApiHook instead
6. Class <class '_hooks.gc_bigquery_hook.BigQueryStandardHook'> incorrectly implements the function run while inheriting from BaseHook. Please make this class inherit from airflow.hooks.db_api_hook.DbApiHook instead
7. Class <class '_hooks.gc_bigquery_hook.BigQueryStandardHook'> incorrectly implements the function get_records while inheriting from BaseHook. Please make this class inherit from airflow.hooks.db_api_hook.DbApiHook instead
8. Class <class 'unusual_prefix_9358643817187292af016429303a09afd4152fc5_gc_bigquery_hook.BigQueryStandardHook'> incorrectly implements the function get_pandas_df while inheriting from BaseHook. Please make this class inherit from airflow.hooks.db_api_hook.DbApiHook instead
9. Class <class 'unusual_prefix_9358643817187292af016429303a09afd4152fc5_gc_bigquery_hook.BigQueryStandardHook'> incorrectly implements the function run while inheriting from BaseHook. Please make this class inherit from airflow.hooks.db_api_hook.DbApiHook instead
10. Class <class 'unusual_prefix_9358643817187292af016429303a09afd4152fc5_gc_bigquery_hook.BigQueryStandardHook'> incorrectly implements the function get_records while inheriting from BaseHook. Please make this class inherit from airflow.hooks.db_api_hook.DbApiHook instead
11. Class <class '_hooks.gc_bigquery_hook.GCBigQueryHook'> incorrectly implements the function get_pandas_df while inheriting from BaseHook. Please make this class inherit from airflow.hooks.db_api_hook.DbApiHook instead
12. Class <class '_hooks.gc_bigquery_hook.GCBigQueryHook'> incorrectly implements the function run while inheriting from BaseHook. Please make this class inherit from airflow.hooks.db_api_hook.DbApiHook instead
13. Class <class '_hooks.gc_bigquery_hook.GCBigQueryHook'> incorrectly implements the function get_records while inheriting from BaseHook. Please make this class inherit from airflow.hooks.db_api_hook.DbApiHook instead
14. Class <class 'unusual_prefix_9358643817187292af016429303a09afd4152fc5_gc_bigquery_hook.GCBigQueryHook'> incorrectly implements the function get_pandas_df while inheriting from BaseHook. Please make this class inherit from airflow.hooks.db_api_hook.DbApiHook instead
15. Class <class 'unusual_prefix_9358643817187292af016429303a09afd4152fc5_gc_bigquery_hook.GCBigQueryHook'> incorrectly implements the function run while inheriting from BaseHook. Please make this class inherit from airflow.hooks.db_api_hook.DbApiHook instead
16. Class <class 'unusual_prefix_9358643817187292af016429303a09afd4152fc5_gc_bigquery_hook.GCBigQueryHook'> incorrectly implements the function get_records while inheriting from BaseHook. Please make this class inherit from airflow.hooks.db_api_hook.DbApiHook instead
The custom hooks all inherit from BigQueryHook
. Looking at the code, it inherits both DbApiHook
and GoogleCloudBaseHook
which inherits BaseHook
. I also noticed that the same BigQueryHook
implementation appears in the 2.0-stable
branch so this is probably a false positive. The warning (1) is already fixed by https://github.com/apache/airflow/pull/14344.
What you expected to happen:
No warnings
How to reproduce it: N/A
Issue Analytics
- State:
- Created 3 years ago
- Comments:8 (5 by maintainers)
Top Results From Across the Web
Upgrade Check Script - Apache Airflow
The Upgrade Check Script is part of a separate Python Package, since it is separate from the core Apache Airflow package and is...
Read more >Airflow Documentation - Read the Docs
Airflow is a platform to programmatically author, schedule and monitor workflows. Use airflow to author workflows as directed acyclic graphs ...
Read more >apache-airflow-upgrade-check - Python package - Snyk
Learn more about apache-airflow-upgrade-check: package health score, popularity, ... in your projects for vulnerabilities and provides automated fix advice.
Read more >Cloud Composer release notes | Google Cloud
(Airflow 1.10.15) Airflow Upgrade Checker updated to version 1.4.0. (Airflow 1.10.15) Fixes in the apache-airflow-backport-providers-google package: ...
Read more >Source code for bigquery_hook - Apache Airflow Documentation
[docs] def get_conn(self): """ Returns a BigQuery PEP 249 connection object. ... :type dataset_id: string :param table_id: The name of the table to...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
PR: https://github.com/apache/airflow/pull/16543
fixed by https://github.com/apache/airflow/pull/16543 and released in 1.4.0