HTTP 2 support
See original GitHub issueI peeked at the docs, at the issues but couldn’t find any info about HTTP 2 support.
Does scrapy
support it?
Issue Analytics
- State:
- Created 8 years ago
- Reactions:3
- Comments:25 (20 by maintainers)
Top Results From Across the Web
HTTP/2 protocol | Can I use... Support tables for HTML5 ...
HTTP /2 is only supported over TLS (HTTPS). See also the precursor of HTTP/2, the SPDY protocol, which has been deprecated and removed...
Read more >HTTP/2 - Wikipedia
Most major browsers had added HTTP/2 support by the end of 2015. ... About 97% of web browsers used have the capability. ......
Read more >HTTP/2 Frequently Asked Questions
In browsers, HTTP/2 is supported by the most current releases of Edge, Safari, Firefox and Chrome. Other browsers based upon Blink will also...
Read more >HTTP/2 Test - Verify HTTP/2 Support
This test can check HTTP and HTTPS but most browsers only support HTTP/2 for HTTPS, which means you must migrate to HTTPS if...
Read more >How to check if a website has HTTP/2 protocol support
Open the browser development tools and switch to the network tab. There you'll see h2 if HTTP/2 ...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Hey @kmike, I’m afraid I disagree. I’m seeing an increase in websites (using a specific vendor) that detect bot traffic based on HTTP2/HTTP1.1 version compared to what’s the expected default from your user agent. While I do understand that that does not make it a priority for the Twisted team, I’m sure it has a significant impact for a lot of scrapy users.
Hi, @Glennvd I have encountered multiple sites that blocked scrapy due to HTTP1.1, but i am finding they are only identifying my spider due to scrapy automatically capitalizing the keys of the headers. I have been able to use a workaround that has worked 100% of the time so far.
example This is what the site expects: accept-encoding: gzip, deflate, br accept-language: en-US,en;q=0.9,hi;q=0.8,pt;q=0.7
And this is what scrapy sends(even if you make them lowercase in the spider): Accept-Encoding: gzip, deflate, br Accept-Language: en-US,en;q=0.9,hi;q=0.8,pt;q=0.7
This seems to be a workaround to keep the keys lowercase:
I hope this helps someone:)