Importing entry_point plugins fails on Airflow v1.10.14 and Python 3.8

Apache Airflow version: 1.10.14 Kubernetes version (if you are using kubernetes) (use kubectl version): N/A Environment:

• Cloud provider or hardware configuration:
• OS (e.g. from /etc/os-release): Debian GNU/Linux 10 (buster)
• Kernel (e.g. uname -a): Linux 3f9779a91397 5.4.0-52-generic #57-Ubuntu SMP Thu Oct 15 10:57:00 UTC 2020 x86_64 GNU/Linux
• Others: Python 3.8

What happened:

On Airflow 1.10.14 and Python 3.8 importing plugins that have been provided through a setuptools entrypoint fails with the following error:

[2020-12-14 13:35:12,200] {{plugins_manager.py:159}} ERROR - Failed to import plugin gantry
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/airflow/plugins_manager.py", line 150, in load_entrypoint_plugins
    plugin_obj.__usable_import_name = entry_point.module
AttributeError: 'EntryPoint' object has no attribute 'module'
/usr/local/lib/python3.8/site-packages/gantry_plugin/plugin.py:11: FutureWarning: Registering operators or sensors in plugins is deprecated -- these should be treated like 'plain' python modules, and imported normally in DAGs.

Airflow 2.0 has removed the ability to register these types in plugins. See <http://airflow.apache.org/docs/stable/howto/custom-operator.html>.
  class GantryPlugin(AirflowPlugin):
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/airflow/plugins_manager.py", line 150, in load_entrypoint_plugins
    plugin_obj.__usable_import_name = entry_point.module
AttributeError: 'EntryPoint' object has no attribute 'module'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/airflow", line 25, in <module>
    from airflow.configuration import conf
  File "/usr/local/lib/python3.8/site-packages/airflow/__init__.py", line 100, in <module>
    operators._integrate_plugins()
  File "/usr/local/lib/python3.8/site-packages/airflow/operators/__init__.py", line 107, in _integrate_plugins
    from airflow.plugins_manager import operators_modules
  File "/usr/local/lib/python3.8/site-packages/airflow/plugins_manager.py", line 229, in <module>
    plugins = load_entrypoint_plugins(
  File "/usr/local/lib/python3.8/site-packages/airflow/plugins_manager.py", line 160, in load_entrypoint_plugins
    import_errors[entry_point.module] = str(e)
AttributeError: 'EntryPoint' object has no attribute 'module'

This plugin can be imported without issues on Airflow v1.10.12 and v1.10.13.

What you expected to happen:

I had expected Airflow to import the plugin without issues.

How to reproduce it:

I believe one should be able to reproduce this by creating a plugin that uses setuptools’ entry_points mechanism as described here.

Anything else we need to know:

I believe this bug may be related to the changes introduced in https://github.com/apache/airflow/commit/b49838fac8b04c32b4aeb211de858e7897d5b3e5. This issue also manifests itself despite of the efforts in https://github.com/apache/airflow/pull/12859.

[potiuk]

Be sure to watch the devlist. It might well be that 1.10.15 will be released at some point in time 😃

[RikHeijdens]

Ah, excellent, I hadn’t noticed that the issue was fixed on the v1-10-test branch. I had only looked at the v1-10-stable branch before opening this issue. I’ll make sure to check that branch before opening an issue the next time.

Either way, thanks for the quick response to this bug report 😃