push: userless ssh url doesn't work

Bug Report

Description

When pushing to a ssh remote which url doesn’t have a user (the user is set in ~/.ssh/config) I get the following error:

ERROR: unexpected error - No existing session

When checking with dvc push --verbose I get the following debug message:

DEBUG: Establishing ssh connection with 'myhost.example' through port '22' as user 'None'

Reproduce

1. dvc init
2. Copy dataset.zip to the directory
3. dvc add dataset.zip
4. dvc remote add --default ssh-remote ssh://myhost.example/path/to/folder
5. Configure ~/.ssh/config with User and IdentityFile for Host myhost

Host myhost.example
    HostName myhost.example
    User myuser
    IdentityFile /path/to/my/private/key

1. dvc push

Expected

My data is pushed to the ssh remote

Environment information

• My computer username is different from the one used with ssh remote

Output of dvc doctor:

$ dvc doctor
DVC version: 2.4.1 (rpm)
---------------------------------
Platform: Python 3.8.10 on Linux-5.12.12-arch1-1-x86_64-with-glibc2.7
Supports: azure, gdrive, gs, webhdfs, http, https, s3, ssh, oss, webdav, webdavs
Cache types: hardlink, symlink
Cache directory: ext4 on /dev/mapper/VolGroup00-lvolhome
Caches: local
Remotes: ssh, ssh
Workspace directory: ext4 on /dev/mapper/VolGroup00-lvolhome
Repo: dvc, git

Additional Information (if any):

$ dvc push --verbose
2021-06-25 15:16:19,871 DEBUG: Check for update is enabled.
2021-06-25 15:16:20,289 DEBUG: Preparing to upload data to 'ssh://myhost.example/path/to/folder'
2021-06-25 15:16:20,289 DEBUG: Preparing to collect status from ssh://myhost.example/path/to/folder
2021-06-25 15:16:20,289 DEBUG: Collecting information from local cache...
2021-06-25 15:16:20,290 DEBUG: Collecting information from remote cache...                          
2021-06-25 15:16:20,291 DEBUG: Matched '0' indexed hashes
2021-06-25 15:16:20,293 DEBUG: Establishing ssh connection with 'myhost.example' through port '22' as user 'None'
2021-06-25 15:16:20,937 ERROR: unexpected error - No existing session                               
------------------------------------------------------------
Traceback (most recent call last):
  File "dvc/fs/pool.py", line 51, in get_connection
IndexError: pop from an empty deque

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "dvc/main.py", line 55, in main
  File "dvc/command/base.py", line 50, in do_run
  File "dvc/command/data_sync.py", line 57, in run
  File "dvc/repo/__init__.py", line 51, in wrapper
  File "dvc/repo/push.py", line 44, in push
  File "dvc/data_cloud.py", line 79, in push
  File "dvc/remote/base.py", line 57, in wrapper
  File "dvc/remote/base.py", line 494, in push
  File "dvc/remote/base.py", line 351, in _process
  File "dvc/remote/base.py", line 195, in _status
  File "dvc/remote/base.py", line 145, in hashes_exist
  File "dvc/objects/db/ssh.py", line 73, in hashes_exist
  File "concurrent/futures/_base.py", line 619, in result_iterator
  File "concurrent/futures/_base.py", line 444, in result
  File "concurrent/futures/_base.py", line 389, in __get_result
  File "concurrent/futures/thread.py", line 57, in run
  File "dvc/objects/db/ssh.py", line 64, in exists_with_progress
  File "dvc/objects/db/ssh.py", line 31, in batch_exists
  File "contextlib.py", line 113, in __enter__
  File "dvc/fs/pool.py", line 11, in get_connection
  File "dvc/fs/pool.py", line 53, in get_connection
  File "dvc/fs/ssh/connection.py", line 59, in __init__
  File "paramiko/client.py", line 435, in connect
  File "paramiko/client.py", line 764, in _auth
  File "paramiko/client.py", line 740, in _auth
  File "paramiko/transport.py", line 1570, in auth_publickey
paramiko.ssh_exception.SSHException: No existing session
------------------------------------------------------------
2021-06-25 15:16:23,055 DEBUG: Version info for developers:
DVC version: 2.4.1 (rpm)
---------------------------------
Platform: Python 3.8.10 on Linux-5.12.12-arch1-1-x86_64-with-glibc2.7
Supports: azure, gdrive, gs, webhdfs, http, https, s3, ssh, oss, webdav, webdavs
Cache types: hardlink, symlink
Cache directory: ext4 on /dev/mapper/VolGroup00-lvolhome
Caches: local
Remotes: ssh, ssh
Workspace directory: ext4 on /dev/mapper/VolGroup00-lvolhome
Repo: dvc, git

Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!
2021-06-25 15:16:23,058 DEBUG: Analytics is enabled.
2021-06-25 15:16:23,297 DEBUG: Trying to spawn '['daemon', '-q', 'analytics', '/tmp/tmpr0kchyzn']'
2021-06-25 15:16:23,300 DEBUG: Spawned '['daemon', '-q', 'analytics', '/tmp/tmpr0kchyzn']'

Workaround:

Overriding url in local config by adding the user (e.g. ssh://[e-mail]/path/to/folder)

Possible lead:

One possible fix to the issue is:

diff --git a/dvc/fs/ssh/__init__.py b/dvc/fs/ssh/__init__.py
index 192e3ef3..622954f6 100644
--- a/dvc/fs/ssh/__init__.py
+++ b/dvc/fs/ssh/__init__.py
@@ -126,9 +126,9 @@ class SSHFileSystem(BaseFileSystem):  # pylint:disable=abstract-method
 
         return get_connection(
             SSHConnection,
-            path_info.host,
-            username=path_info.user,
-            port=path_info.port,
+            self.host,
+            username=self.user,
+            port=self.port,
             key_filename=self.keyfile,
             timeout=self.timeout,
             password=self.password,

However I don’t know if it breaks other actions.

[Erotemic]

I’m also running into this issue.

I verified in dvc/remote/base.py in Remote.__init__ the **config does indeed contain the “user” field, but in dvc/fs/ssh/connection.py in SSHConnection.__Init__, the **kwargs does not contain “username”.

I’m not sure if the difference between “user” and “username” is important or not.

In dvc/fs/ssh/__init__.py would a solution being changing:

    def ssh(self, path_info):
        self.ensure_credentials()

        from .connection import SSHConnection

        return get_connection(
            SSHConnection,
            path_info.host,
            username=path_info.user,
            port=path_info.port,
            key_filename=self.keyfile,
            timeout=self.timeout,
            password=self.password,
            gss_auth=self.gss_auth,
            sock=self.sock,
            allow_agent=self.allow_agent,
        )

to

    def ssh(self, path_info):
        self.ensure_credentials()

        from .connection import SSHConnection

        return get_connection(
            SSHConnection,
            path_info.host,
            username=self.user,
            port=path_info.port,
            key_filename=self.keyfile,
            timeout=self.timeout,
            password=self.password,
            gss_auth=self.gss_auth,
            sock=self.sock,
            allow_agent=self.allow_agent,
        )

Basically username=path_info.user, -> username=self.user,. The self.user variable seems to have the correct information in it.

[pmrowla]

Looks like it’s related to the changes in https://github.com/iterative/dvc/pull/6059

Previously, fs.path_info was populated using self.user, but that was lost when path_info was removed from fs/remote into the ODB classes