Stuck on an issue?
Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
Make brotlipy a hard Scrapy dependency
See original GitHub issueWhen we added support to Scrapy (https://github.com/scrapy/scrapy/pull/2535), we did so without making brotlipy a hard dependency, seemingly because there seem to be some potential for installation issues.
However, given that most web browsers support Brotli (br) compression out of the box, I think we should consider revisiting that. It would also slightly simplify our code.
Related to https://github.com/scrapy/scrapy/issues/4697
Issue Analytics
- State:
- Created 3 years ago
- Reactions:1
- Comments:6 (6 by maintainers)
Top Results From Across the Web
Installation guide — Scrapy 2.7.1 documentation
(See platform-specific guides below for non-Python dependencies that you may need to install beforehand). Platform specific installation notes¶.
Read more >Release notes — Scrapy 2.7.1 documentation
To enable HttpCompressionMiddleware to do brotli compression, installing brotli is now recommended instead of installing brotlipy, as the former provides a ...
Read more >latest PDF - Scrapy Documentation
Scrapy (/skrepa/) is an application framework for crawling web sites and extracting structured data which can be used.
Read more >1.6 PDF - Scrapy Documentation
Scrapy is an application framework for crawling web sites and extracting structured data which can be used for a wide.
Read more >Scrapy Tutorial — Scrapy 2.7.1 documentation
Learn Python 3 The Hard Way ... They must subclass Spider and define the initial requests to make, optionally how to follow links...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
How about doing this in 2.7?
It seems brotli is popular and maintained (e.g. there are wheels uploaded recently), even if last release was in 2020. That’s probably fine to add it as a dependency. On the other hand, it’s still a possible source of install errors in various environments, as it’s a binary package. So, no strong opinion on that.