deploy fails with python2.7 with updated pip and upgraded awsebcli

Description

Suddenly deploy fails due to inability to import JSONDecodeError in ebcli/core/fileoperations.py", line 32. Previously the deployment script worked fine, but right now there are probably some unexpected updates in dependencies.

Steps to reproduce

Start a docker image node:14 (as per documentation). In this instance run few updates as following:

For python version 2.7:

apt-get update -y
curl https://bootstrap.pypa.io/pip/2.7/get-pip.py -o get-pip.py
python get-pip.py
pip --version
pip install --upgrade pip setuptools
pip install awsebcli -q --upgrade
eb deploy production

For python version 3.5:

apt-get update -y
apt-get install -y python3-dev python3-pip
update-alternatives --install /usr/bin/python python /usr/bin/python2.7 1
update-alternatives --install /usr/bin/python python /usr/bin/python3.5 2
update-alternatives --list python
python --version
pip3 --version
pip3 install awsebcli -q --upgrade
eb deploy production

Observed result

Please provide command output with --debug flag set.

NOTE: the --debug flag does not produce any difference.

For python version 2.7:

$ eb deploy production
Traceback (most recent call last):
  File "/usr/local/bin/eb", line 5, in <module>
    from ebcli.core.ebcore import main
  File "/usr/local/lib/python2.7/dist-packages/ebcli/core/ebcore.py", line 19, in <module>
    from ebcli.core import ebglobals, base, hooks
  File "/usr/local/lib/python2.7/dist-packages/ebcli/core/hooks.py", line 20, in <module>
    from ebcli.core import fileoperations
  File "/usr/local/lib/python2.7/dist-packages/ebcli/core/fileoperations.py", line 32, in <module>
    from json import load, JSONDecodeError
ImportError: cannot import name JSONDecodeError

For python3:

$ pip3 install awsebcli -q --upgrade
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-0rv71d9c/awsebcli/

Expected result

Normal deploy as before

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

.elasticbeanstalk config contains:

default_platform: Docker 18.06.1-ce

[eyeb14ck]

Until this is fixed one solution is to use previous version: pip install awsebcli==3.19.4 --upgrade

Also haven’t tried myself, but devs from another project had the same problem with python 3.7, they solved the issue by using different python image: image: nikolaik/python-nodejs:latest

[adamantike]

After investigating about this, the bug was first introduced in https://github.com/aws/aws-elastic-beanstalk-cli/commit/6275d6970364be3c15f0fe3f72d06f4242277fe6, indeed included in release 3.20.0. That commit is the first one to add from json import JSONDecodeError, which is only valid since Python 3.5.

To maintain compatibility with Python versions < 3.5, considering JSONDecodeError is a subclass of ValueError, these usages should be converted to:

try:
    from json import JSONDecodeError
except ImportError:
    JSONDecodeError = ValueError