dvc pull ignores ssh-agent keys

Bug Report

pull: ignores ssh-agent keys

Description

DVC 2.10.0 stops using ssh-agent (passed via SSH_AUTH_SOCK variable) for git clone operation resulting in Permission denied error.

Traceback (most recent call last):
[...]
  File "C:\Users\runneradmin\AppData\Local\pypoetry\Cache\virtualenvs\gall-UFKFKqae-py3.8\lib\site-packages\asyncssh\connection.py", line 2262, in wait_established

    await self._waiter

asyncssh.misc.PermissionDenied: Permission denied
[...]

Reproduce

- name: Setup SSH Keys and known_hosts
  env:
    SSH_AUTH_SOCK: /tmp/ssh_agent.sock
  run: |
    ssh-agent -a $SSH_AUTH_SOCK > /dev/null
    ssh-add - <<< "${{ secrets.SSH_KEY }}"
    mkdir ~/.ssh/
    ssh-keyscan -H github.com > ~/.ssh/known_hosts

- name: "Pull latest model"
  run: dvc pull
  env:
    SSH_AUTH_SOCK: /tmp/ssh_agent.sock
    AZURE_STORAGE_CONNECTION_STRING: <connection_string>

Expected

Artefacts are downloaded as in <=2.9.5 version

Environment information

GitHub Actions

Current runner version: '2.291.1'
Operating System
  Microsoft Windows Server 2022
  10.0.20348
  Datacenter
Virtual Environment
  Environment: windows-2022
  Version: 20220503.1
  Included Software: https://github.com/actions/virtual-environments/blob/win22/20220503.1/images/win/Windows2022-Readme.md
  Image Release: https://github.com/actions/virtual-environments/releases/tag/win22%2F20220503.1

[mdekstrand]

@daavoo Yep, seeing that issue is how I knew the HTTPS side was a dulwich problem.

At least until that PR lands in a published release, could dvc get an option to support the older git-based cloning? Dropping support for authentication methods seems like a pretty big workflow regression. Happy to create a separate issue for that if appropriate.

[daavoo]

@daavoo Yep, seeing that issue is how I knew the HTTPS side was a dulwich problem.

At least until that PR lands in a published release, could dvc get an option to support the older git-based cloning?

The only way (disclaimer: it’s hacky 😅 ) I can think of achieving this without requiring a new release (either dvc, scmrepo or dulwich) would be for you to edit your local scmrepo source code, altering the priority of the backends.

https://github.com/iterative/scmrepo/blob/94a772799812f42bf8adacdc4fdf5dc9808f528c/scmrepo/git/__init__.py#L38-L42

I think it might be better to wait for the release or downgrade dvc

Dropping support for authentication methods seems like a pretty big workflow regression. Happy to create a separate issue for that if appropriate.

I think having the ability to select a git backend would be a valid feature request for scmrepo. However, I believe that the current direction in DVC is to focus on using/supporting only dulwich.