question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

py27: dvc push to SSH throws TypeError

See original GitHub issue

Python 2.7 DVC version: 0.52.1

To reproduce:

virtualenv --python=python2.7 .venv
pip install 'dvc[ssh]'
dvc init --no-scm --force
dvc remote add ssh ssh://<your-ssh>
for x in {1..3}; do echo $x > "data/${x}.txt"; done
dvc add data
dvc push -r ssh

Output:

Preparing to upload data to 'ssh://mroutis@104.248.78.4/home/mroutis/cache'
Preparing to collect status from ssh://mroutis@104.248.78.4/home/mroutis/cache
Collecting information from local cache...
[##############################] 100%

Collecting information from remote cache...
[##############################] 100%
[##############################] 100% Analysing status
[                              ] ?% data/3.txt
ERROR: failed to upload '.dvc/cache/6d/7fce9fee471194aa8b5b6e47267f03' to 'ssh://mroutis@104.248.78.4/home/mroutis/cache/6d/7fce9fee471194aa8b5b6e47267f03' - ChannelException(1, 'Administratively prohibited')

Having any troubles?. Hit us up at https://dvc.org/support, we are always happy to help!
[##############################] 100% data/2.txt
[##############################] 100% data
[##############################] 100% data/1.txt
[##############################] 100% data/2.txt
[##############################] 100% data
[##############################] 100% data/1.txt

ERROR: failed to push data to the cloud - 1 file(s) failed to upload

Having any troubles?. Hit us up at https://dvc.org/support, we are always happy to help!
Exception TypeError: 'super() argument 1 must be type, not None' in <bound method SSHPool.__del__ of <dvc.remote.ssh.pool.SSHPool object at 0x7f9b346f2050>> ignored

Then, if I do dvc push -r ssh again to push the remaining file (data/3.txt):

Preparing to upload data to 'ssh://mroutis@104.248.78.4/home/mroutis/cache'
Preparing to collect status from ssh://mroutis@104.248.78.4/home/mroutis/cache
Collecting information from local cache...
[##############################] 100%

Collecting information from remote cache...
[##############################] 100%
[##############################] 100% Analysing status
[##############################] 100% data/3.txt
[##############################] 100% data/3.txt
Exception Unhandled exception in thread started by
Traceback (most recent call last):
TypeError: 'super(  File "/usr/lib64/python2.7/threading.py", line 774, in __bootstrap
) argument 1 must be type, not None'    self.__bootstrap_inner()
  File "/usr/lib64/python2.7/threading.py", line 814, in __bootstrap_inner
     in <bound method SSHPool.__del__ of <dvc.remote.ssh.pool.SSHPool object at 0x7fc043a70710>> ignored
(self.name, _format_exc()))
  File "/usr/lib64/python2.7/traceback.py", line 241, in format_exc
    etype, value, tb = sys.exc_info()
AttributeError: 'NoneType' object has no attribute 'exc_info'

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Comments:6 (5 by maintainers)

github_iconTop GitHub Comments

1reaction
Suorcommented, Jul 18, 2019

Since confusing messages and error hiding is resolved in #2288 and the original issue is a misconfigured SSH, I close it in favor or #2292.

1reaction
darabicommented, Jul 17, 2019

The underlying problem is an error opening the sftp connection which is clearly visible when I use paramiko directly:

Traceback (most recent call last):
  File "sftp_stat.py", line 24, in <module>
    sftp = paramiko.SFTPClient.from_transport(transport)
  File "/opt/python-env/python3/lib/python3.6/site-packages/paramiko/sftp_client.py", line 169, in from_transport
    chan.invoke_subsystem("sftp")
  File "/opt/python-env/python3/lib/python3.6/site-packages/paramiko/channel.py", line 72, in _check
    return func(self, *args, **kwds)
  File "/opt/python-env/python3/lib/python3.6/site-packages/paramiko/channel.py", line 283, in invoke_subsystem
    self._wait_for_event()
  File "/opt/python-env/python3/lib/python3.6/site-packages/paramiko/channel.py", line 1226, in _wait_for_event
    raise e
paramiko.ssh_exception.SSHException: Channel closed.

It would be nice to have all exceptions surface at the command line and of course batch_exists should not continue in case of an sftp error.

With that SSHException right in front of me, I would have realised that my server is not configured for sftp (I implicitly assumed that dvc would use scp for file transfer).

Read more comments on GitHub >

github_iconTop Results From Across the Web

DVC push to SSH remote ERROR: No such file
I do the Getting started guide. where ws is a hostname of a workstation with the SSH access enabled, and hddb/data/dvc-tutorial is the...
Read more >
vocab.txt - Hugging Face
... detail ##lin ##respon typeerror conv ##64 plug indices dataframe endpoint ... google ##items targets actual duplic ##chronous ##ql scalar ssh ##no dyn ......
Read more >
Permission denied when pushing file to remote - Stack Overflow
the user and email used to make commits have nothing to do with the remote repo (here hosted on GitHub) for authentication ·...
Read more >
DVC - Chieh's Blog
Also, suppose if we add one large file by git and push to registry ... In my case, I was stored the data...
Read more >
Dvc Ssh - :: Anaconda.org
linux-64 v2.18.0; noarch v2.20.0; osx-64 v2.18.0; win-64 v2.18.0. conda install. To install this package run one of the following:
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found