Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Connection reset error

See original GitHub issue

I just wanted to share the stacktrace for an error I’ve been getting.

It seems like just catching requests.exceptions.ChunkedEncodingError in catch_conn_reset would be a good idea.

hydrating 2020-03/coronavirus-tweet-id-2020-03-16-04.txt
rate limit exceeded: sleeping 540.7469525337219 secs
caught connection error ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer')) on 1 try
hydrating 2020-03/coronavirus-tweet-id-2020-03-16-05.txt
hydrating 2020-03/coronavirus-tweet-id-2020-03-16-06.txt
rate limit exceeded: sleeping 540.9134702682495 secs
hydrating 2020-03/coronavirus-tweet-id-2020-03-16-07.txt
Traceback (most recent call last):
  File "/homes/thielk/envs/misc/lib/python3.6/site-packages/urllib3/response.py", line 362, in _error_catcher
    yield
  File "/homes/thielk/envs/misc/lib/python3.6/site-packages/urllib3/response.py", line 444, in read
    data = self._fp.read(amt)
  File "/usr/lib/python3.6/http/client.py", line 459, in read
    n = self.readinto(b)
  File "/usr/lib/python3.6/http/client.py", line 503, in readinto
    n = self.fp.readinto(b)
  File "/usr/lib/python3.6/socket.py", line 586, in readinto
    return self._sock.recv_into(b)
  File "/usr/lib/python3.6/ssl.py", line 1012, in recv_into
    return self.read(nbytes, buffer)
  File "/usr/lib/python3.6/ssl.py", line 874, in read
    return self._sslobj.read(len, buffer)
  File "/usr/lib/python3.6/ssl.py", line 631, in read
    v = self._sslobj.read(len, buffer)
ConnectionResetError: [Errno 104] Connection reset by peer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/homes/thielk/envs/misc/lib/python3.6/site-packages/requests/models.py", line 750, in generate
    for chunk in self.raw.stream(chunk_size, decode_content=True):
  File "/homes/thielk/envs/misc/lib/python3.6/site-packages/urllib3/response.py", line 496, in stream
    data = self.read(amt=amt, decode_content=decode_content)
  File "/homes/thielk/envs/misc/lib/python3.6/site-packages/urllib3/response.py", line 461, in read
    raise IncompleteRead(self._fp_bytes_read, self.length_remaining)
  File "/usr/lib/python3.6/contextlib.py", line 99, in __exit__
    self.gen.throw(type, value, traceback)
  File "/homes/thielk/envs/misc/lib/python3.6/site-packages/urllib3/response.py", line 380, in _error_catcher
    raise ProtocolError('Connection broken: %r' % e, e)
urllib3.exceptions.ProtocolError: ("Connection broken: ConnectionResetError(104, 'Connection reset by peer')", ConnectionResetError(104, 'Connection reset by peer'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "hydrate.py", line 42, in <module>
    main()
  File "hydrate.py", line 25, in main
    hydrate(path)
  File "hydrate.py", line 37, in hydrate
    for tweet in twarc.hydrate(id_file.open()):
  File "/homes/thielk/envs/misc/lib/python3.6/site-packages/twarc/client.py", line 486, in hydrate
    "include_ext_alt_text": 'true'
  File "/homes/thielk/envs/misc/lib/python3.6/site-packages/twarc/decorators.py", line 17, in new_f
    resp = f(*args, **kwargs)
  File "/homes/thielk/envs/misc/lib/python3.6/site-packages/twarc/decorators.py", line 77, in new_f
    return f(self, *args, **kwargs)
  File "/homes/thielk/envs/misc/lib/python3.6/site-packages/twarc/decorators.py", line 87, in new_f
    return f(self, *args, **kwargs)
  File "/homes/thielk/envs/misc/lib/python3.6/site-packages/twarc/decorators.py", line 103, in new_f
    return f(self, *args, **kwargs)
  File "/homes/thielk/envs/misc/lib/python3.6/site-packages/twarc/client.py", line 729, in post
    **kwargs)
  File "/homes/thielk/envs/misc/lib/python3.6/site-packages/requests/sessions.py", line 581, in post
    return self.request('POST', url, data=data, json=json, **kwargs)
  File "/homes/thielk/envs/misc/lib/python3.6/site-packages/requests/sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "/homes/thielk/envs/misc/lib/python3.6/site-packages/requests/sessions.py", line 686, in send
    r.content
  File "/homes/thielk/envs/misc/lib/python3.6/site-packages/requests/models.py", line 828, in content
    self._content = b''.join(self.iter_content(CONTENT_CHUNK_SIZE)) or b''
  File "/homes/thielk/envs/misc/lib/python3.6/site-packages/requests/models.py", line 753, in generate
    raise ChunkedEncodingError(e)
requests.exceptions.ChunkedEncodingError: ("Connection broken: ConnectionResetError(104, 'Connection reset by peer')", ConnectionResetError(104, 'Connection reset by peer'))