question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Allow setting `chunk_size` for `Response.iter_bytes()` etc...

See original GitHub issue

Requests allowed setting chunk_size within .iter_content() which is currently not an option for our alternatives .stream() and .stream_text().

For .stream_text() we should go the extra step and fix the issue that users sometimes run into when using this feature and use chunk-size for measuring the decoded text, not the raw bytes.

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Reactions:5
  • Comments:11 (10 by maintainers)

github_iconTop GitHub Comments

2reactions
florimondmancacommented, Mar 13, 2020

@b0g3r Careful that we’re in a sort of transition state w.r.t. urllib3 usage due to #804 (we’ll soon use our own sync implementation, though keeping urllib3 as an option). Due to this I wouldn’t advise relying on any existing urllib3 functionality — also because we’d want to provide chunk sizing on the async layer too, and it’d be odd to have a different implementation on both sides.

I think we want to look at controlling the chunk size directly from response.iter_bytes()/response.aiter_bytes(), instead…

2reactions
florimondmancacommented, Mar 12, 2020

Hi @b0g3r! I think this is still something we’d like to have, and given discussions in https://github.com/python-gitlab/python-gitlab/pull/1036 it seems like some folks would like to see it too. 😃

Ways to move forward would be:

  • Propose an API for this, with context on the existing API on Requests
  • Investigate implementation details (ie how are we going to split chunks: buffering, other? Looking at how Requests/urllib3 do this can help)
  • Draft a PR 😃
Read more comments on GitHub >

github_iconTop Results From Across the Web

Ideal Chunk Size for python requests - Stack Overflow
So as long as you aren't very concerned about keeping memory usage down, go ahead and specify a large chunk size, such as...
Read more >
2014 - Stupid Python Ideas
You've probably been told that you can convert any recursive function into an iterative loop just by using an explicit stack. Tail-recursive functions....
Read more >
[PATCH v2 00/21] blksnap - block devices snapshots module
Removed default values for module parameters from the configuration file. ... + * + * The bdev_filter_detach() function allows to detach the ...
Read more >
2012-July.txt - Python mailing list
A toc would let people who have found this doc know to look for >> this at the ... we are calling the...
Read more >
mozilla-central: changeset 424282 ...
This allows attribute order to be - preserved through html5lib if the ... This file is execfile()d with the current directory set to...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found