Lightrun
question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

get: throwing 2 network errors

See original GitHub issue

Bug Report

Description

I’m randomly getting two different errors when I try to use dvc get to download the cats-dogs example dataset. They seem to be network issues.

Sometimes the terminal gets spammed with the following:

AttributeError: 'RetryClient' object has no attribute '_connector'

And sometimes I get the following:

ERROR: unexpected error - Cannot connect to host http://s3-us-east-2.amazonaws.com:443/ ssl:default [nodename nor servname provided, or not known]: [Errno 8] nodename nor servname provided, or not known

This is happening in the latest release and the development version.

Reproduce

dvc get https://github.com/iterative/dataset-registry use-cases/cats-dogs

To re-run, I just rm -rf cats-dogs and then do it again.

Expected

I expect the command to finish with no errors and data to be in the cats-dogs folder.

Environment information

This was happening for me on macOS BigSur 11.6 with the brew installed version and a version installed in an Anaconda environment.

Output of dvc doctor:

DVC version: 2.7.4 (osxpkg)
---------------------------------
Platform: Python 3.8.12 on macOS-10.16-x86_64-i386-64bit
Supports:
	azure (adlfs = 2021.9.1, knack = 0.8.2, azure-identity = 1.6.1),
	gdrive (pydrive2 = 1.9.4),
	gs (gcsfs = 2021.8.1),
	hdfs (pyarrow = 5.0.0),
	webhdfs (hdfs = 2.6.0),
	http (aiohttp = 3.7.4.post0, aiohttp-retry = 2.4.5),
	https (aiohttp = 3.7.4.post0, aiohttp-retry = 2.4.5),
	s3 (s3fs = 2021.8.1, boto3 = 1.17.106),
	ssh (sshfs = 2021.8.1),
	oss (ossfs = 2021.8.0),
	webdav (webdav4 = 0.9.2),
	webdavs (webdav4 = 0.9.2)

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Reactions:1
  • Comments:6 (6 by maintainers)

github_iconTop GitHub Comments

3reactions
isidenticalcommented, Oct 1, 2021

It seems like this was due to asyncio-retry not providing a private method that fsspec sometimes uses (this is why sometimes it is not reproducable) (when there is a timeout). I already sent a PR against fsspec, and will be resolved with the new release (hopefully today) https://github.com/intake/filesystem_spec/pull/779

1reaction
dberenbaumcommented, Sep 29, 2021

I’m able to reproduce with the latest dev version. It appears to have been introduced in #6565. Here’s the full output:

$ dvc -v get https://github.com/iterative/dataset-registry use-cases/cats-dogs
  1% Downloading directory|▎                         |31/2800 [00:02<02:12, 20.92Files/s]Exception ignored in: <finalize object at 0x11021e360; dead>
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/weakref.py", line 580, in __call__
    return info.func(*info.args, **(info.kwargs or {}))
  File "/opt/homebrew/lib/python3.9/site-packages/fsspec/implementations/http.py", line 126, in close_session
    if session._connector is not None:
AttributeError: 'RetryClient' object has no attribute '_connector'
Exception ignored in: <finalize object at 0x11021e320; dead>
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/weakref.py", line 580, in __call__
    return info.func(*info.args, **(info.kwargs or {}))
  File "/opt/homebrew/lib/python3.9/site-packages/fsspec/implementations/http.py", line 126, in close_session
    if session._connector is not None:
AttributeError: 'RetryClient' object has no attribute '_connector'
Exception ignored in: <finalize object at 0x11021e300; dead>
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/weakref.py", line 580, in __call__
    return info.func(*info.args, **(info.kwargs or {}))
  File "/opt/homebrew/lib/python3.9/site-packages/fsspec/implementations/http.py", line 126, in close_session
    if session._connector is not None:
AttributeError: 'RetryClient' object has no attribute '_connector'
Exception ignored in: <finalize object at 0x11021e2e0; dead>
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/weakref.py", line 580, in __call__
    return info.func(*info.args, **(info.kwargs or {}))
  File "/opt/homebrew/lib/python3.9/site-packages/fsspec/implementations/http.py", line 126, in close_session
    if session._connector is not None:
AttributeError: 'RetryClient' object has no attribute '_connector'
Exception ignored in: <finalize object at 0x11021e2c0; dead>
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/weakref.py", line 580, in __call__
    return info.func(*info.args, **(info.kwargs or {}))
  File "/opt/homebrew/lib/python3.9/site-packages/fsspec/implementations/http.py", line 126, in close_session
    if session._connector is not None:
AttributeError: 'RetryClient' object has no attribute '_connector'
Exception ignored in: <finalize object at 0x11021e0c0; dead>
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/weakref.py", line 580, in __call__
    return info.func(*info.args, **(info.kwargs or {}))
  File "/opt/homebrew/lib/python3.9/site-packages/fsspec/implementations/http.py", line 126, in close_session
    if session._connector is not None:
AttributeError: 'RetryClient' object has no attribute '_connector'
Exception ignored in: <finalize object at 0x11021e080; dead>
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/weakref.py", line 580, in __call__
    return info.func(*info.args, **(info.kwargs or {}))
  File "/opt/homebrew/lib/python3.9/site-packages/fsspec/implementations/http.py", line 126, in close_session
    if session._connector is not None:
AttributeError: 'RetryClient' object has no attribute '_connector'
Exception ignored in: <finalize object at 0x110296ba0; dead>
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/weakref.py", line 580, in __call__
    return info.func(*info.args, **(info.kwargs or {}))
  File "/opt/homebrew/lib/python3.9/site-packages/fsspec/implementations/http.py", line 126, in close_session
    if session._connector is not None:
AttributeError: 'RetryClient' object has no attribute '_connector'
Exception ignored in: <finalize object at 0x110296c40; dead>
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/weakref.py", line 580, in __call__
    return info.func(*info.args, **(info.kwargs or {}))
  File "/opt/homebrew/lib/python3.9/site-packages/fsspec/implementations/http.py", line 126, in close_session
    if session._connector is not None:
AttributeError: 'RetryClient' object has no attribute '_connector'
Exception ignored in: <finalize object at 0x110296ca0; dead>
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/weakref.py", line 580, in __call__
    return info.func(*info.args, **(info.kwargs or {}))
  File "/opt/homebrew/lib/python3.9/site-packages/fsspec/implementations/http.py", line 126, in close_session
    if session._connector is not None:
AttributeError: 'RetryClient' object has no attribute '_connector'
Exception ignored in: <finalize object at 0x110296cc0; dead>
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/weakref.py", line 580, in __call__
    return info.func(*info.args, **(info.kwargs or {}))
  File "/opt/homebrew/lib/python3.9/site-packages/fsspec/implementations/http.py", line 126, in close_session
    if session._connector is not None:
AttributeError: 'RetryClient' object has no attribute '_connector'
Exception ignored in: <finalize object at 0x110296ce0; dead>
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/weakref.py", line 580, in __call__
    return info.func(*info.args, **(info.kwargs or {}))
  File "/opt/homebrew/lib/python3.9/site-packages/fsspec/implementations/http.py", line 126, in close_session
    if session._connector is not None:
AttributeError: 'RetryClient' object has no attribute '_connector'
Exception ignored in: <finalize object at 0x110296d00; dead>
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/weakref.py", line 580, in __call__
    return info.func(*info.args, **(info.kwargs or {}))
  File "/opt/homebrew/lib/python3.9/site-packages/fsspec/implementations/http.py", line 126, in close_session
    if session._connector is not None:
AttributeError: 'RetryClient' object has no attribute '_connector'
Exception ignored in: <finalize object at 0x110296d20; dead>
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/weakref.py", line 580, in __call__
    return info.func(*info.args, **(info.kwargs or {}))
  File "/opt/homebrew/lib/python3.9/site-packages/fsspec/implementations/http.py", line 126, in close_session
    if session._connector is not None:
AttributeError: 'RetryClient' object has no attribute '_connector'
Exception ignored in: <finalize object at 0x110296d60; dead>
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/weakref.py", line 580, in __call__
    return info.func(*info.args, **(info.kwargs or {}))
  File "/opt/homebrew/lib/python3.9/site-packages/fsspec/implementations/http.py", line 126, in close_session
    if session._connector is not None:
AttributeError: 'RetryClient' object has no attribute '_connector'
Exception ignored in: <finalize object at 0x110296da0; dead>
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/weakref.py", line 580, in __call__
    return info.func(*info.args, **(info.kwargs or {}))
  File "/opt/homebrew/lib/python3.9/site-packages/fsspec/implementations/http.py", line 126, in close_session
    if session._connector is not None:
AttributeError: 'RetryClient' object has no attribute '_connector'
  1% Downloading directory|▎                         |33/2800 [00:19<00:59, 46.32Files/s]Exception ignored in: <finalize object at 0x110296dc0; dead>
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/weakref.py", line 580, in __call__
    return info.func(*info.args, **(info.kwargs or {}))
  File "/opt/homebrew/lib/python3.9/site-packages/fsspec/implementations/http.py", line 126, in close_session
    if session._connector is not None:
AttributeError: 'RetryClient' object has no attribute '_connector'
Exception ignored in: <finalize object at 0x110296e00; dead>
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/weakref.py", line 580, in __call__
    return info.func(*info.args, **(info.kwargs or {}))
  File "/opt/homebrew/lib/python3.9/site-packages/fsspec/implementations/http.py", line 126, in close_session
    if session._connector is not None:
AttributeError: 'RetryClient' object has no attribute '_connector'
Exception ignored in: <finalize object at 0x110296e40; dead>
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/weakref.py", line 580, in __call__
    return info.func(*info.args, **(info.kwargs or {}))
  File "/opt/homebrew/lib/python3.9/site-packages/fsspec/implementations/http.py", line 126, in close_session
    if session._connector is not None:
AttributeError: 'RetryClient' object has no attribute '_connector'
Exception ignored in: <finalize object at 0x110296e60; dead>
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/weakref.py", line 580, in __call__
    return info.func(*info.args, **(info.kwargs or {}))
  File "/opt/homebrew/lib/python3.9/site-packages/fsspec/implementations/http.py", line 126, in close_session
    if session._connector is not None:
AttributeError: 'RetryClient' object has no attribute '_connector'
Exception ignored in: <finalize object at 0x110296e80; dead>
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/weakref.py", line 580, in __call__
    return info.func(*info.args, **(info.kwargs or {}))
  File "/opt/homebrew/lib/python3.9/site-packages/fsspec/implementations/http.py", line 126, in close_session
    if session._connector is not None:
AttributeError: 'RetryClient' object has no attribute '_connector'
Exception ignored in: <finalize object at 0x110296ea0; dead>
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/weakref.py", line 580, in __call__
    return info.func(*info.args, **(info.kwargs or {}))
  File "/opt/homebrew/lib/python3.9/site-packages/fsspec/implementations/http.py", line 126, in close_session
    if session._connector is not None:
AttributeError: 'RetryClient' object has no attribute '_connector'
Exception ignored in: <finalize object at 0x110296ec0; dead>
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/weakref.py", line 580, in __call__
    return info.func(*info.args, **(info.kwargs or {}))
  File "/opt/homebrew/lib/python3.9/site-packages/fsspec/implementations/http.py", line 126, in close_session
    if session._connector is not None:
AttributeError: 'RetryClient' object has no attribute '_connector'
Exception ignored in: <finalize object at 0x110296ee0; dead>
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/weakref.py", line 580, in __call__
    return info.func(*info.args, **(info.kwargs or {}))
  File "/opt/homebrew/lib/python3.9/site-packages/fsspec/implementations/http.py", line 126, in close_session
    if session._connector is not None:
AttributeError: 'RetryClient' object has no attribute '_connector'
Exception ignored in: <finalize object at 0x110296f00; dead>
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/weakref.py", line 580, in __call__
    return info.func(*info.args, **(info.kwargs or {}))
  File "/opt/homebrew/lib/python3.9/site-packages/fsspec/implementations/http.py", line 126, in close_session
    if session._connector is not None:
AttributeError: 'RetryClient' object has no attribute '_connector'
Exception ignored in: <finalize object at 0x110296f20; dead>
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/weakref.py", line 580, in __call__
    return info.func(*info.args, **(info.kwargs or {}))
  File "/opt/homebrew/lib/python3.9/site-packages/fsspec/implementations/http.py", line 126, in close_session
    if session._connector is not None:
AttributeError: 'RetryClient' object has no attribute '_connector'
Exception ignored in: <finalize object at 0x110296f40; dead>
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/weakref.py", line 580, in __call__
    return info.func(*info.args, **(info.kwargs or {}))
  File "/opt/homebrew/lib/python3.9/site-packages/fsspec/implementations/http.py", line 126, in close_session
    if session._connector is not None:
AttributeError: 'RetryClient' object has no attribute '_connector'
Exception ignored in: <finalize object at 0x110296f60; dead>
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/weakref.py", line 580, in __call__
    return info.func(*info.args, **(info.kwargs or {}))
  File "/opt/homebrew/lib/python3.9/site-packages/fsspec/implementations/http.py", line 126, in close_session
    if session._connector is not None:
AttributeError: 'RetryClient' object has no attribute '_connector'
Exception ignored in: <finalize object at 0x110296320; dead>
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/weakref.py", line 580, in __call__
    return info.func(*info.args, **(info.kwargs or {}))
  File "/opt/homebrew/lib/python3.9/site-packages/fsspec/implementations/http.py", line 126, in close_session
    if session._connector is not None:
AttributeError: 'RetryClient' object has no attribute '_connector'
Exception ignored in: <finalize object at 0x1102962e0; dead>
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/weakref.py", line 580, in __call__
    return info.func(*info.args, **(info.kwargs or {}))
  File "/opt/homebrew/lib/python3.9/site-packages/fsspec/implementations/http.py", line 126, in close_session
    if session._connector is not None:
AttributeError: 'RetryClient' object has no attribute '_connector'
Exception ignored in: <finalize object at 0x1102965c0; dead>
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/weakref.py", line 580, in __call__
    return info.func(*info.args, **(info.kwargs or {}))
  File "/opt/homebrew/lib/python3.9/site-packages/fsspec/implementations/http.py", line 126, in close_session
    if session._connector is not None:
AttributeError: 'RetryClient' object has no attribute '_connector'
Exception ignored in: <finalize object at 0x110296600; dead>
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/weakref.py", line 580, in __call__
    return info.func(*info.args, **(info.kwargs or {}))
  File "/opt/homebrew/lib/python3.9/site-packages/fsspec/implementations/http.py", line 126, in close_session
    if session._connector is not None:
AttributeError: 'RetryClient' object has no attribute '_connector'
Exception ignored in: <finalize object at 0x11091b7e0; dead>
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/weakref.py", line 580, in __call__
    return info.func(*info.args, **(info.kwargs or {}))
  File "/opt/homebrew/lib/python3.9/site-packages/fsspec/implementations/http.py", line 126, in close_session
    if session._connector is not None:
AttributeError: 'RetryClient' object has no attribute '_connector'
ERROR: unexpected error - Server disconnected

Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!
Read more comments on GitHub >

github_iconTop Results From Across the Web

Handling operation errors - Apollo GraphQL Docs
When a network error occurs, Apollo Client adds it to the error.networkError field returned by your useQuery call (or whichever operation hook you...
Read more >
React: Axios Network Error - Stack Overflow
I received a network error with axios 0.27.2 when I was trying to upload an image to our server. After I set headers...
Read more >
Fix "Aw, Snap!" page crashes and other page loading errors
Step 1: Check your internet connection. Make sure your computer's connected to Wi-Fi or a wired network. Try reloading the tab with the...
Read more >
[SOLVED] How to Fix the ERR_CONNECTION_TIMED_OUT ...
Ways to Fix Err_Connection_Timed_Out Error · Method 1: Try VPN · Method 2: Flush DNS Cache · Method 3: Check Your Connection ·...
Read more >
Fix JavaScript errors that are reported in the Console
In the Console, the following two errors occurred: A network error that starts with GET HTTP method followed by a URI.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found
Previous page

diff: ERROR: unexpected error - 'not in cache'

Next page

dvc pull returns "failed to pull data" when the data exists on remote