Can not get certificate information when http body is empty.
See original GitHub issueDescription
Can not get certificate information when http body is empty.
Steps to Reproduce
Here is the Code:
# -*- coding: utf-8 -*-
import scrapy
class TestSpider(scrapy.Spider):
name = 'test'
# 49.235.192.63:443
start_urls = ['https://49.235.192.63/', ]
handle_httpstatus_list = [i for i in range(199, 600)] # handle all response
def parse(self, response):
if response.certificate:
cert = response.certificate.inspect()
else:
cert = ""
print("Cert is --->", cert)
Expected behavior: [What you expect to happen] cert is not empty Actual behavior: [What actually happens] cert is empty Reproduces how often: [What percentage of the time does it reproduce?] every time
Versions
Scrapy : 2.0.1 lxml : 4.5.0.0 libxml2 : 2.9.10 cssselect : 1.1.0 parsel : 1.5.2 w3lib : 1.21.0 Twisted : 20.3.0 Python : 3.7.5 (default, Nov 20 2019, 09:21:52) - [GCC 9.2.1 20191008] pyOpenSSL : 19.1.0 (OpenSSL 1.1.1d 10 Sep 2019) cryptography : 2.8 Platform : Linux-5.3.0-45-generic-x86_64-with-Ubuntu-19.10-eoan
Additional context
this connectionMade won’t be called when the response has nobody. I do not know how to resolve and need your help. https://github.com/scrapy/scrapy/blob/dfbe1d95071acfbba159cea051530749b1684460/scrapy/core/downloader/handlers/http11.py#L467 Thanks so much! #4054
Issue Analytics
- State:
- Created 3 years ago
- Comments:8 (6 by maintainers)
Top GitHub Comments
I reproduced this behavior of
protocol.transport.getPeerCertificate()
I mentioned above in a fairly short piece of code and created a ticket in twisted issue tracker.This is probably not a beginner-friendly issue.