ansible_python_interpreter: "{{ '/usr/bin/env python-docker' }}" and python3

Hi,

I’ve got two servers:

• infra1: running Ubuntu 20.04 LTS, defaults to python3, by using the workaround I mentioned in #82
• infra2: running Ubuntu 18.04 LTS, defaults to python2

When running the very same ansible playbook, I’m running into issues while trying to configure private docker registries on 20.04 (as no previous step makes use of ansible_python_interpreter: "{{ '/usr/bin/env python-docker' }}", but all which do suffer the same issue):

TASK [nickjj.docker : Manage Docker registry login credentials] *********************************************************************************************************************************************************************failed: [infra1] (item={u'username': u'***', u'password': u'***', u'registry_url': u'****'}) => {"ansible_loop_var": "item", "changed": false, "item": {"password": "***", "registry_url": "****", "username": "***"}, "msg": "Failed to import the required Python library (Docker SDK for Python: docker (Python >= 2.7) or docker-py (Python 2.6)) on infra1's Python /usr/local/bin/python-docker. Please read module documentation and install in the appropriate location. If the required library is installed, but Ansible is using the wrong Python interpreter, please consult the documentation on ansible_python_interpreter, for example via `pip install docker` or `pip install docker-py` (Python 2.6). The error was: No module named 'docker'"}
changed: [infra2] => (item={u'username': u'***', u'password': u'***', u'registry_url': u'****'})
failed: [infra1] (item={u'username': u'***', u'password': u'***', u'config_path': u'***/.docker/config.json', u'registry_url': u'****'}) => {"ansible_loop_var": "item", "changed": false, "item": {"config_path": "***/.docker/config.json", "password": "***", "registry_url": "****", "username": "***"}, "msg": "Failed to import the required Python library (Docker SDK for Python: docker (Python >= 2.7) or docker-py (Python 2.6)) on infra1's Python /usr/local/bin/python-docker. Please read module documentation and install in the appropriate location. If the required library is installed, but Ansible is using the wrong Python interpreter, please consult the documentation on ansible_python_interpreter, for example via `pip install docker` or `pip install docker-py` (Python 2.6). The error was: No module named 'docker'"}
changed: [infra2] => (item={u'username': u'***', u'password': u'***', u'config_path': u'***/.docker/config.json', u'registry_url': u'****'})

This made me curious and I wanted to dig deeper:

infra2 (python 2, Ubuntu 18.04 LTS) -> OK

$ ll /usr/local/bin/python-docker
lrwxrwxrwx 1 root root 43 May 11 22:32 /usr/local/bin/python-docker -> /usr/local/lib/docker/virtualenv/bin/python*
$ ll /usr/local/lib/docker/virtualenv/bin/python
lrwxrwxrwx 1 root root 7 May 11 22:31 /usr/local/lib/docker/virtualenv/bin/python -> python2*
$ python-docker
Python 2.7.17 (default, Apr 15 2020, 17:20:14)
[GCC 7.5.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> print(sys.path)
['', '/usr/local/lib/docker/virtualenv/lib/python2.7', '/usr/local/lib/docker/virtualenv/lib/python2.7/plat-x86_64-linux-gnu', '/usr/local/lib/docker/virtualenv/lib/python2.7/lib-tk', '/usr/local/lib/docker/virtualenv/lib/python2.7/lib-old', '/usr/local/lib/docker/virtualenv/lib/python2.7/lib-dynload', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-x86_64-linux-gnu', '/usr/lib/python2.7/lib-tk', '/usr/local/lib/docker/virtualenv/local/lib/python2.7/site-packages', '/usr/local/lib/docker/virtualenv/lib/python2.7/site-packages']
>>> import docker
>>> 

infra1 (python 3, Ubuntu 20.04 LTS) -> NOK

$ which python-docker
/usr/local/bin/python-docker
$ ll /usr/local/bin/python-docker
lrwxrwxrwx 1 root root 43 May 11 22:32 /usr/local/bin/python-docker -> /usr/local/lib/docker/virtualenv/bin/python*
$ ll /usr/local/lib/docker/virtualenv/bin/python
lrwxrwxrwx 1 root root 16 May 11 22:31 /usr/local/lib/docker/virtualenv/bin/python -> /usr/bin/python3*
$ python-docker
Python 3.8.2 (default, Apr 27 2020, 15:53:34)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> print(sys.path)
['', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages']
>>> import docker
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'docker'
>>> exit()
$ source /usr/local/lib/docker/virtualenv/bin/activate
(virtualenv) $ python
Python 3.8.2 (default, Apr 27 2020, 15:53:34) 
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import docker
>>> import sys
>>> print(sys.path)
['', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/usr/local/lib/docker/virtualenv/lib/python3.8/site-packages']

So it seems that either the change to python3 or Ubuntu 20 changed the way they are importing paths in virtual environments: the /usr/local/lib/docker/virtualenv/lib/python3.8/site-packages are only included when I activate the venv directly. I know it’s not necessarily an issue with this ansible role, but have you experienced something similar before? (I’ll also continue searching of course, but not tonight 😄)

Thanks!

[nickjj]

Yeah I think removing it might not be the right play. Ideally I’d like to not change anything from the role / documentation level and get things to work the same as they did before except instead of making a symlink, you can make a proxy script.

Basically a bash script that does nothing except call the full binary path but pass in $@ to forward any arguments. I’ll take care of this PR, don’t worry about it.

[shadow1runner]

Thanks for your work on this, I’ve just tried with your master branch and it is working fine 👍