ImportError: cannot import name 'bytes_type' from 'oauthlib.common'

Apache Airflow version: 1.10.10 Environment: Airflow ECS

• Cloud provider or hardware configuration: Running on Docker
• OS (e.g. from /etc/os-release): AWS Linux 2
• Kernel (e.g. uname -a): Linux
• Install tools:
• Others:

I have made some change in Dockerfile and added these lines

ARG WEBSERVER_CONFIG="default_webserver_config.py"
ENV WEBSERVER_CONFIG=${WEBSERVER_CONFIG}

COPY --chown=airflow:airflow "./airflow/config_templates/${WEBSERVER_CONFIG}" ${AIRFLOW_HOME}/webserver_config.py

ARG AIRFLOW_CFG="default_airflow.cfg"
ENV AIRFLOW_CFG=${AIRFLOW_CFG}

COPY --chown=airflow:airflow "./airflow/config_templates/${AIRFLOW_CFG}" ${AIRFLOW_HOME}/airflow.cfg

What happened I am updating airflow from 1.10.5 to 1.10.10. I am getting the following error in webserver when using google OAuth.

Traceback (most recent call last):
  File "/home/airflow/.local/bin/airflow", line 37, in <module>
    args.func(args)
  File "/home/airflow/.local/lib/python3.7/site-packages/airflow/utils/cli.py", line 75, in wrapper
    return f(*args, **kwargs)
  File "/home/airflow/.local/lib/python3.7/site-packages/airflow/bin/cli.py", line 900, in webserver
    app = cached_app_rbac(None) if settings.RBAC else cached_app(None)
  File "/home/airflow/.local/lib/python3.7/site-packages/airflow/www_rbac/app.py", line 295, in cached_app
    app, _ = create_app(config, session, testing)
  File "/home/airflow/.local/lib/python3.7/site-packages/airflow/www_rbac/app.py", line 108, in create_app
    update_perms=conf.getboolean('webserver', 'UPDATE_FAB_PERMS'))
  File "/home/airflow/.local/lib/python3.7/site-packages/flask_appbuilder/base.py", line 148, in __init__
    self.init_app(app, session)
  File "/home/airflow/.local/lib/python3.7/site-packages/flask_appbuilder/base.py", line 202, in init_app
    self.sm = self.security_manager_class(self)
  File "/home/airflow/.local/lib/python3.7/site-packages/airflow/www_rbac/security.py", line 177, in __init__
    super(AirflowSecurityManager, self).__init__(appbuilder)
  File "/home/airflow/.local/lib/python3.7/site-packages/flask_appbuilder/security/sqla/manager.py", line 51, in __init__
    super(SecurityManager, self).__init__(appbuilder)
  File "/home/airflow/.local/lib/python3.7/site-packages/flask_appbuilder/security/manager.py", line 249, in __init__
    from flask_oauthlib.client import OAuth
  File "/home/airflow/.local/lib/python3.7/site-packages/flask_oauthlib/client.py", line 20, in <module>
    from .utils import to_bytes
  File "/home/airflow/.local/lib/python3.7/site-packages/flask_oauthlib/utils.py", line 5, in <module>
    from oauthlib.common import to_unicode, bytes_type
ImportError: cannot import name 'bytes_type' from 'oauthlib.common' (/home/airflow/.local/lib/python3.7/site-packages/oauthlib/common.py)

This error is because of the dependency mismatch. Airflow 1.10.10 is installing oauthlib==3.1.0 and Flask-OAuthlib==0.9.5 and in 3.1.0 version of oauthlib, there is no bytes_type while Flash-OAuthlib is expecting it to be there.

Considering that lepture has archived flask-oauthlib and created authlib, I see a PR in airflow from lepture about this change https://github.com/apache/airflow/pull/6140 but it is not merged and marked stale by the bot

Has anyone faced this issue in Airflow 1.10.10? I have fixed this issue by changing the oauthlib version to 2.1.0 in requirement.txt

[namiops]

hey @namiops what version of airflow are you using? and python version?

hi @snigdhasaikam30, I’m using Airfllow 1.10.15 with python 3.8.

[snigdhasaikam30]

@snigdhasaikam30 try using --constraint=https://raw.githubusercontent.com/apache/airflow/cc382b75783d76888fcd5b0e38228f08ee83dbd5/constraints-3.6.txt when running the pip install command.

That constraint is the one that is working for me that I have in #14562. I’m using the same python version and airflow version that you are.

This is what worked for us. changed the user to “airflow” instead of root

USER airflow RUN pip install --user -I flask_oauthlib==0.9.6 --constraint=https://raw.githubusercontent.com/apache/airflow/cc382b75783d76888fcd5b0e38228f08ee83dbd5/constraints-3.6.txt

airflow:1.10.15-python3.6