Docker Agent can't use SSH for --base-url

Description

Docker Agent can’t connect to remote Docker daemon with SSH. It results in

Traceback (most recent call last):
  File "C:\Users\tfh\Miniconda3\envs\prefect\lib\site-packages\docker\api\client.py", line 165, in __init__
    self._custom_adapter = SSHHTTPAdapter(
NameError: name 'SSHHTTPAdapter' is not defined

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\tfh\Miniconda3\envs\prefect\Scripts\prefect-script.py", line 9, in <module>
    sys.exit(cli())
  File "C:\Users\tfh\Miniconda3\envs\prefect\lib\site-packages\click\core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "C:\Users\tfh\Miniconda3\envs\prefect\lib\site-packages\click\core.py", line 782, in main
    rv = self.invoke(ctx)
  File "C:\Users\tfh\Miniconda3\envs\prefect\lib\site-packages\click\core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "C:\Users\tfh\Miniconda3\envs\prefect\lib\site-packages\click\core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "C:\Users\tfh\Miniconda3\envs\prefect\lib\site-packages\click\core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\Users\tfh\Miniconda3\envs\prefect\lib\site-packages\click\core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "C:\Users\tfh\Miniconda3\envs\prefect\lib\site-packages\click\decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "C:\Users\tfh\Miniconda3\envs\prefect\lib\site-packages\prefect\cli\agent.py", line 260, in start
    docker_interface=not no_docker_interface,
  File "C:\Users\tfh\Miniconda3\envs\prefect\lib\site-packages\prefect\agent\docker\agent.py", line 117, in __init__
    self.docker_client = self._get_docker_client()
  File "C:\Users\tfh\Miniconda3\envs\prefect\lib\site-packages\prefect\agent\docker\agent.py", line 142, in _get_docker_client
    return docker.APIClient(base_url=self.base_url, version="auto")
  File "C:\Users\tfh\Miniconda3\envs\prefect\lib\site-packages\docker\api\client.py", line 170, in __init__
    'Install paramiko package to enable ssh:// support'
docker.errors.DockerException: Install paramiko package to enable ssh:// support

Expected Behavior

That I could connect to docker host.

Reproduction

Simply run: prefect agent start docker --base-url ssh://mydockerhost

Environment

(prefect) λ prefect diagnostics { "config_overrides": { "backend": true, "server": { "endpoint": true, "graphql": { "host": true }, "host": true, "host_port": true, "port": true, "ui": { "endpoint": true, "graphql_url": true, "host": true, "host_port": true, "port": true } } }, "env_vars": [], "system_information": { "platform": "Windows-10-10.0.18362-SP0", "prefect_version": "0.11.5", "python_version": "3.7.0" } }

[joshmeek]

Thanks for opening @thomasfrederikhoeck! Will take a look at this. One thing, if you pip install paramiko then try running this again does it start working?

[lauralorenz]

Glad to see that you got it worked out 🎉 Closing this now that we got it figured out.

To follow up on your questions: on my windows machine, I have prefect installed via pip from pypi, and then also from my local source using pip; out of curiosity I also installed it just now in another environment using conda instead (and followed the same package manager, respectively, for when I installed paramiko). I didn’t experience the cryptography problems from any of the 3 situations.

All that being said, it is up to the user whether they install their dependencies using pip or conda. cryptography is a dependency of paramiko, not prefect (since docker chose not to make paramiko a hard dependency). If the speculation in that linked spacy google group is true and cryptography has issues with the Anaconda distro of python and pip, then it depends how the user install paramiko in particular if they will experience this issue with cryptography; and ultimately it is an upstream problem that is potentially best served by a ticket on the cryptography repo.