Proxy Tunnel Error message length
See original GitHub issueSummary
Proxy TunnelError message length shouldn’t be truncated to 32 bytes
Motivation
Often there might be some Proxy TunnelErrors which are currently truncated to 32 characters by these lines of code: https://github.com/scrapy/scrapy/blame/master/scrapy/core/downloader/handlers/http11.py#L170 / https://github.com/scrapy/scrapy/blame/master/scrapy/core/downloader/handlers/http11.py#L125
Describe alternatives you’ve considered
Is there a legitimate reason why this should be truncated?
Additional context
I’m getting this exception raised
scrapy.core.downloader.handlers.http11.TunnelError: Could not open CONNECT tunnel with proxy zproxy.lum-superproxy.io:22225 [{'status': 502, 'reason': b'Proxy Error: server_error No pe'}]
instead of
scrapy.core.downloader.handlers.http11.TunnelError: Could not open CONNECT tunnel with proxy zproxy.lum-superproxy.io:22225 [{'status': 502, 'reason': b'Proxy Error: server_error No peers available'}]
Tried out by removing the 32 character truncation in my specific use-case and it seems to work.
Issue Analytics
- State:
- Created 3 years ago
- Comments:7 (6 by maintainers)
Top GitHub Comments
1K? 😃
@akinad3 I don’t think the length of 64 will be short for any error message here. Should I make the change?