Stuck on an issue?
Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
iter_lines small chunks
See original GitHub issueI’m writing a script to track keywords in the twitter realtime api.
At first, I was using requests v0.10.8. Showing the tweets was almost instant, but the issue #515 occured frequently.
Then I upgraded requests to v0.11.1. Now #515 is resolved, but apparently small chunks are not processed instantly, if they don’t reach the chunk_size. By setting the chunk size to a value like 10, it works, but that can’t be the solution 😃
Issue Analytics
- State:
- Created 11 years ago
- Comments:15 (7 by maintainers)
Top Results From Across the Web
iter_lines() sometimes yields broken results when chunk_size ...
Here is a small test program to demonstrate the bug: import requests ... the data from different chunks rather than limiting max line...
Read more >How to iterate over a list in chunks - python - Stack Overflow
import itertools def chunks(iterable,size): it = iter(iterable) chunk = tuple(itertools.islice(it,size)) while chunk: yield chunk chunk ...
Read more >Iterating in fixed-size chunks in Python - alexwlchan
A snippet for iterating over an arbitrary iterable in chunks, and returning a smaller chunk if the boundaries don't line up.
Read more >zoem - Micans
In this mode, zoem interprets by default chunks of text that are ended by a single dot on a line of its own....
Read more >Iterating non-newline-separated files should be easier
The problem is that readline is hardcoded to look for b'\n' for binary files, smart-universal-newline-thingy for text files, there's no way to ...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Well, if
read(chunk_size)blocks until at leastchunk_sizebytes are read then the whole implementation does not make sense and changingchunk_sizeto 1 is not the solution.That makes sense; we’d need our own event loop, and it probably wouldn’t play nice with the
geventevent loop.