question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Airflow upgrade_check gives false positives on DbApi functions implementations check

See original GitHub issue

Apache Airflow version: 1.10.14

Kubernetes version (if you are using kubernetes) (use kubectl version): N/A

Environment:

  • Cloud provider or hardware configuration: GCP
  • OS (e.g. from /etc/os-release): Ubuntu 18.04.4 LTS
  • Kernel (e.g. uname -a): N/A
  • Install tools: pip

What happened:

Running airflow upgrade_check threw the following warnings:

  1.  Class <class 'airflow.hooks.http_hook.HttpHook'> incorrectly implements the function run while inheriting from BaseHook. Please make this class inherit from airflow.hooks.db_api_hook.DbApiHook instead
  2.  Class <class 'airflow.contrib.hooks.bigquery_hook.BigQueryHook'> incorrectly implements the function get_pandas_df while inheriting from BaseHook. Please make this class inherit from airflow.hooks.db_api_hook.DbApiHook instead
  3.  Class <class 'airflow.contrib.hooks.bigquery_hook.BigQueryHook'> incorrectly implements the function run while inheriting from BaseHook. Please make this class inherit from airflow.hooks.db_api_hook.DbApiHook instead
  4.  Class <class 'airflow.contrib.hooks.bigquery_hook.BigQueryHook'> incorrectly implements the function get_records while inheriting from BaseHook. Please make this class inherit from airflow.hooks.db_api_hook.DbApiHook instead
  5.  Class <class '_hooks.gc_bigquery_hook.BigQueryStandardHook'> incorrectly implements the function get_pandas_df while inheriting from BaseHook. Please make this class inherit from airflow.hooks.db_api_hook.DbApiHook instead
  6.  Class <class '_hooks.gc_bigquery_hook.BigQueryStandardHook'> incorrectly implements the function run while inheriting from BaseHook. Please make this class inherit from airflow.hooks.db_api_hook.DbApiHook instead
  7.  Class <class '_hooks.gc_bigquery_hook.BigQueryStandardHook'> incorrectly implements the function get_records while inheriting from BaseHook. Please make this class inherit from airflow.hooks.db_api_hook.DbApiHook instead
  8.  Class <class 'unusual_prefix_9358643817187292af016429303a09afd4152fc5_gc_bigquery_hook.BigQueryStandardHook'> incorrectly implements the function get_pandas_df while inheriting from BaseHook. Please make this class inherit from airflow.hooks.db_api_hook.DbApiHook instead
  9.  Class <class 'unusual_prefix_9358643817187292af016429303a09afd4152fc5_gc_bigquery_hook.BigQueryStandardHook'> incorrectly implements the function run while inheriting from BaseHook. Please make this class inherit from airflow.hooks.db_api_hook.DbApiHook instead
 10.  Class <class 'unusual_prefix_9358643817187292af016429303a09afd4152fc5_gc_bigquery_hook.BigQueryStandardHook'> incorrectly implements the function get_records while inheriting from BaseHook. Please make this class inherit from airflow.hooks.db_api_hook.DbApiHook instead
 11.  Class <class '_hooks.gc_bigquery_hook.GCBigQueryHook'> incorrectly implements the function get_pandas_df while inheriting from BaseHook. Please make this class inherit from airflow.hooks.db_api_hook.DbApiHook instead
 12.  Class <class '_hooks.gc_bigquery_hook.GCBigQueryHook'> incorrectly implements the function run while inheriting from BaseHook. Please make this class inherit from airflow.hooks.db_api_hook.DbApiHook instead
 13.  Class <class '_hooks.gc_bigquery_hook.GCBigQueryHook'> incorrectly implements the function get_records while inheriting from BaseHook. Please make this class inherit from airflow.hooks.db_api_hook.DbApiHook instead
 14.  Class <class 'unusual_prefix_9358643817187292af016429303a09afd4152fc5_gc_bigquery_hook.GCBigQueryHook'> incorrectly implements the function get_pandas_df while inheriting from BaseHook. Please make this class inherit from airflow.hooks.db_api_hook.DbApiHook instead
 15.  Class <class 'unusual_prefix_9358643817187292af016429303a09afd4152fc5_gc_bigquery_hook.GCBigQueryHook'> incorrectly implements the function run while inheriting from BaseHook. Please make this class inherit from airflow.hooks.db_api_hook.DbApiHook instead
 16.  Class <class 'unusual_prefix_9358643817187292af016429303a09afd4152fc5_gc_bigquery_hook.GCBigQueryHook'> incorrectly implements the function get_records while inheriting from BaseHook. Please make this class inherit from airflow.hooks.db_api_hook.DbApiHook instead

The custom hooks all inherit from BigQueryHook. Looking at the code, it inherits both DbApiHook and GoogleCloudBaseHook which inherits BaseHook. I also noticed that the same BigQueryHook implementation appears in the 2.0-stable branch so this is probably a false positive. The warning (1) is already fixed by https://github.com/apache/airflow/pull/14344.

What you expected to happen:

No warnings

How to reproduce it: N/A

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:8 (5 by maintainers)

github_iconTop GitHub Comments

2reactions
kaxilcommented, Jun 19, 2021
0reactions
eladkalcommented, Jun 26, 2021

fixed by https://github.com/apache/airflow/pull/16543 and released in 1.4.0

Read more comments on GitHub >

github_iconTop Results From Across the Web

Upgrade Check Script - Apache Airflow
The Upgrade Check Script is part of a separate Python Package, since it is separate from the core Apache Airflow package and is...
Read more >
Airflow Documentation - Read the Docs
Airflow is a platform to programmatically author, schedule and monitor workflows. Use airflow to author workflows as directed acyclic graphs ...
Read more >
apache-airflow-upgrade-check - Python package - Snyk
Learn more about apache-airflow-upgrade-check: package health score, popularity, ... in your projects for vulnerabilities and provides automated fix advice.
Read more >
Cloud Composer release notes | Google Cloud
(Airflow 1.10.15) Airflow Upgrade Checker updated to version 1.4.0. (Airflow 1.10.15) Fixes in the apache-airflow-backport-providers-google package: ...
Read more >
Source code for bigquery_hook - Apache Airflow Documentation
[docs] def get_conn(self): """ Returns a BigQuery PEP 249 connection object. ... :type dataset_id: string :param table_id: The name of the table to...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found