Docker API support issue on version 4.3.0

Issue:

It appears that docker-py (pip’s package docker) version 1.4.0 is erroring out with the following:

Traceback (most recent call last):
  File "/home/travis/virtualenv/python3.8.1/lib/python3.8/site-packages/docker/api/client.py", line 261, in _raise_for_status
    response.raise_for_status()
  File "/home/travis/virtualenv/python3.8.1/lib/python3.8/site-packages/requests/models.py", line 941, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: http+docker://localhost/v1.39/_ping
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/home/travis/virtualenv/python3.8.1/bin/molecule", line 8, in <module>
    sys.exit(main())
  File "/home/travis/virtualenv/python3.8.1/lib/python3.8/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/travis/virtualenv/python3.8.1/lib/python3.8/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/travis/virtualenv/python3.8.1/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/travis/virtualenv/python3.8.1/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/travis/virtualenv/python3.8.1/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/travis/virtualenv/python3.8.1/lib/python3.8/site-packages/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/travis/virtualenv/python3.8.1/lib/python3.8/site-packages/molecule/command/test.py", line 147, in test
    base.execute_cmdline_scenarios(scenario_name, args, command_args)
  File "/home/travis/virtualenv/python3.8.1/lib/python3.8/site-packages/molecule/command/base.py", line 105, in execute_cmdline_scenarios
    execute_scenario(scenario)
  File "/home/travis/virtualenv/python3.8.1/lib/python3.8/site-packages/molecule/command/base.py", line 147, in execute_scenario
    execute_subcommand(scenario.config, action)
  File "/home/travis/virtualenv/python3.8.1/lib/python3.8/site-packages/molecule/command/base.py", line 136, in execute_subcommand
    return command(config).execute()
  File "/home/travis/virtualenv/python3.8.1/lib/python3.8/site-packages/molecule/command/destroy.py", line 107, in execute
    self._config.provisioner.destroy()
  File "/home/travis/virtualenv/python3.8.1/lib/python3.8/site-packages/molecule/provisioner/ansible.py", line 699, in destroy
    pb.execute()
  File "/home/travis/virtualenv/python3.8.1/lib/python3.8/site-packages/molecule/provisioner/ansible_playbook.py", line 104, in execute
    self._config.driver.sanity_checks()
  File "/home/travis/virtualenv/python3.8.1/lib/python3.8/site-packages/molecule/driver/docker.py", line 235, in sanity_checks
    docker_client.ping()
  File "/home/travis/virtualenv/python3.8.1/lib/python3.8/site-packages/docker/client.py", line 187, in ping
    return self.api.ping(*args, **kwargs)
  File "/home/travis/virtualenv/python3.8.1/lib/python3.8/site-packages/docker/api/daemon.py", line 166, in ping
    return self._result(self._get(self._url('/_ping'))) == 'OK'
  File "/home/travis/virtualenv/python3.8.1/lib/python3.8/site-packages/docker/api/client.py", line 267, in _result
    self._raise_for_status(response)
  File "/home/travis/virtualenv/python3.8.1/lib/python3.8/site-packages/docker/api/client.py", line 263, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/home/travis/virtualenv/python3.8.1/lib/python3.8/site-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.APIError: 400 Client Error: Bad Request ("client version 1.39 is too new. Maximum supported API version is 1.38")

OS:

Ubuntu 18.04 GCP instance.

Docker Version output:

Client:
 Version:           18.06.0-ce
 API version:       1.38
 Go version:        go1.10.3
 Git commit:        0ffa825
 Built:             Wed Jul 18 19:09:54 2018
 OS/Arch:           linux/amd64
 Experimental:      false

Server:
 Engine:
  Version:          18.06.0-ce
  API version:      1.38 (minimum version 1.12)
  Go version:       go1.10.3
  Git commit:       0ffa825
  Built:            Wed Jul 18 19:07:56 2018
  OS/Arch:          linux/amd64
  Experimental:     false

docker-py version:

4.3.0

As you can see above the docker API verison is 1.38 but the error states that 1.39 is not supported. You can review the build here.

Temporary workaround:

If I switch docker-py back to 4.2.2 (for e.g. pip3 install docker==4.2.2 ) - it works out fine. Example of the build here.

[samdoran]

I was able to get around this by specifying dist: focal in the .travis.yml. Either way, it seems the solution is to use a newer version of Docker. Not sure that is a bug in docker-py, per se. Sometimes you just gotta update your stuff. 😁

[darkwizard242]

@aiordache @thaJeztah - Thanks for responding to this GitHub issue.

Frankly, it’s not that I was purposely managing an older version of docker engine but rather that TravisCI was. Basically, I have a multiplebuilds that run on TravisCI to build/test ansible roles using molecule, docker-py, testinfra and other things.

Considering that the temporary build instance that provisions out by TravisCI on GCP has a pre-baked Image (Ubuntu 18.04) containing an older docker version, I went ahead and updated my build configuration to remove any pre-existing version of Docker-CE and install latest stable version (currently install Client/Server version 19.03.12 and API 1.40). Fortunately, that did not cause any issues with docker-py 1.4.0 and the builds were able to run successfully.

For anyone else that comes across this issue, uses TravisCI instances and utilizes docker engine (older versions) as well as docker-py… Please ensure that you have a before_install section to perform the following:

before_install:
  - sudo apt-get update
  - sudo apt-get upgrade -y
  - sudo apt-get purge docker-ce -y
  - sudo apt-get autoremove -y
  - curl -fsSL https://raw.githubusercontent.com/darkwizard242/devopsubuntu1804/v1.0/packer-build/mods/10-docker.sh | sudo bash -s -- install
  - sudo usermod -aG docker $USER
  - curl -fsSL https://raw.githubusercontent.com/darkwizard242/devopsubuntu1804/v1.0/packer-build/mods/10-docker.sh | sudo bash -s -- restart

• sudo apt-get purge docker-ce -y - will remove docker-ce that comes in built in TravisCI.
• curl -fsSL https://raw.githubusercontent.com/darkwizard242/devopsubuntu1804/v1.0/packer-build/mods/10-docker.sh | sudo bash -s – install - will install latest stable version of docker-ce (only for Ubuntu 16.04 and Ubuntu 18.04).
• sudo usermod -aG docker $USER - will add travisci build user to docker group.
• curl -fsSL https://raw.githubusercontent.com/darkwizard242/devopsubuntu1804/v1.0/packer-build/mods/10-docker.sh | sudo bash -s – restart - will restart docker systemd service.

Well, anyone that matches the same issue and runs an older docker version - can utilize a more latest stable docker-ce version with docker-py for a quick resolution.