How to handle IncompleteRead errors while streaming

Everytime I run my code for a long time, I get that type of error:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/http/client.py", line 555, in _get_chunk_left
    chunk_left = self._read_next_chunk_size()
  File "/usr/local/lib/python3.8/http/client.py", line 522, in _read_next_chunk_size
    return int(line, 16)
ValueError: invalid literal for int() with base 16: b''

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/http/client.py", line 587, in _readinto_chunked
    chunk_left = self._get_chunk_left()
  File "/usr/local/lib/python3.8/http/client.py", line 557, in _get_chunk_left
    raise IncompleteRead(b'')
http.client.IncompleteRead: IncompleteRead(0 bytes read)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/urllib3/response.py", line 436, in _error_catcher
    yield
  File "/usr/local/lib/python3.8/site-packages/urllib3/response.py", line 518, in read
    data = self._fp.read(amt) if not fp_closed else b""
  File "/usr/local/lib/python3.8/http/client.py", line 458, in read
    n = self.readinto(b)
  File "/usr/local/lib/python3.8/http/client.py", line 492, in readinto
    return self._readinto_chunked(b)
  File "/usr/local/lib/python3.8/http/client.py", line 603, in _readinto_chunked
    raise IncompleteRead(bytes(b[0:total_bytes]))
http.client.IncompleteRead: IncompleteRead(2 bytes read)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.8/site-packages/tweepy/streaming.py", line 320, in _run
    six.reraise(*exc_info)
  File "/usr/local/lib/python3.8/site-packages/six.py", line 703, in reraise
    raise value
  File "/usr/local/lib/python3.8/site-packages/tweepy/streaming.py", line 289, in _run
    self._read_loop(resp)
  File "/usr/local/lib/python3.8/site-packages/tweepy/streaming.py", line 339, in _read_loop
    line = buf.read_line()
  File "/usr/local/lib/python3.8/site-packages/tweepy/streaming.py", line 200, in read_line
    self._buffer += self._stream.read(self._chunk_size)
  File "/usr/local/lib/python3.8/site-packages/urllib3/response.py", line 540, in read
    raise IncompleteRead(self._fp_bytes_read, self.length_remaining)
  File "/usr/local/lib/python3.8/contextlib.py", line 131, in __exit__
    self.gen.throw(type, value, traceback)
  File "/usr/local/lib/python3.8/site-packages/urllib3/response.py", line 454, in _error_catcher
    raise ProtocolError("Connection broken: %r" % e, e)
urllib3.exceptions.ProtocolError: ('Connection broken: IncompleteRead(2 bytes read)', IncompleteRead(2 bytes read))

I already try to edit streaming.py like here

I have no idea how to fix it, any help is welcomed

[jamsspinle]

I’ve encountered this problem quite a few times and came up with a work around. I created an asynchronous function that runs in the background and checks every so often if the stream is still running. If it’s not, it will create another. It looks something like this:

async def listen():
  stream = #createStream, make sure to create the stream so it's async.
  
  while(True):
    if not stream.running:
      await asyncio.sleep(5)
      stream = #createStreamAgain

    else:
      await asyncio.sleep(5) 

The sleep statement immediately after the if statement may not be needed, but I added it incase the API got mad that another stream was created so quick. Hopefully this helps you.

[Harmon758]

Applying individual commits from the master branch to an older version is ill-advised, as there can likely be context missing from additional prior or subsequent commits on the master branch. Regardless, even if https://github.com/tweepy/tweepy/commit/68e19cc6b9b23d72369ca1520093770eb18a5a9f was applied to v3.10 in the relevant parts of the code, that traceback would not be possible, since the error being raised is handled with that commit.

Manually modifying Tweepy itself is not reproducible and will not be supported. Instead, as noted, if you really need a change that hasn’t been released yet, you should install the development version on the master branch, but there are other breaking changes currently on the master branch that you’ll need to take into consideration. Otherwise, you should see the referenced issues for the likely cause of this issue.

Feel free to create a new issue if you’re still able to reproduce this using the master branch.