question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Can not get certificate information when http body is empty.

See original GitHub issue

Description

Can not get certificate information when http body is empty.

Steps to Reproduce

Here is the Code:

# -*- coding: utf-8 -*-
import scrapy
class TestSpider(scrapy.Spider):
    name = 'test'
    # 49.235.192.63:443
    start_urls = ['https://49.235.192.63/', ]
    handle_httpstatus_list = [i for i in range(199, 600)]  # handle all response

    def parse(self, response):
        if response.certificate:
            cert = response.certificate.inspect()
        else:
            cert = ""
        print("Cert is --->", cert)

Expected behavior: [What you expect to happen] cert is not empty Actual behavior: [What actually happens] cert is empty Reproduces how often: [What percentage of the time does it reproduce?] every time

Versions

Scrapy : 2.0.1 lxml : 4.5.0.0 libxml2 : 2.9.10 cssselect : 1.1.0 parsel : 1.5.2 w3lib : 1.21.0 Twisted : 20.3.0 Python : 3.7.5 (default, Nov 20 2019, 09:21:52) - [GCC 9.2.1 20191008] pyOpenSSL : 19.1.0 (OpenSSL 1.1.1d 10 Sep 2019) cryptography : 2.8 Platform : Linux-5.3.0-45-generic-x86_64-with-Ubuntu-19.10-eoan

Additional context

this connectionMade won’t be called when the response has nobody. I do not know how to resolve and need your help. https://github.com/scrapy/scrapy/blob/dfbe1d95071acfbba159cea051530749b1684460/scrapy/core/downloader/handlers/http11.py#L467 Thanks so much! #4054

Issue Analytics

  • State:open
  • Created 3 years ago
  • Comments:8 (6 by maintainers)

github_iconTop GitHub Comments

3reactions
michalp2213commented, May 31, 2020

I reproduced this behavior of protocol.transport.getPeerCertificate() I mentioned above in a fairly short piece of code and created a ticket in twisted issue tracker.

1reaction
Gallaeciocommented, Apr 1, 2020

This is probably not a beginner-friendly issue.

Read more comments on GitHub >

github_iconTop Results From Across the Web

“HTTP request body must not be empty” on build.ps1 ... - MSDN
Hi, When we run our Connected Factory build (via build.ps1) for Azure, we run into the error below. The FullyQualifiedErrorId is "Microsoft.
Read more >
Problem with empty body in HTTP Request
Problem with empty body in HTTP Request. I think I found a bug. I tried to get information about my Dropbox account: (documentation)...
Read more >
OkHttpClient with certificate return empty body, when I try with ...
Here I create a OkHttpClient with client certificate and private key, seems that everything is ok but please if someone can check this....
Read more >
How to Send an Empty Body for HTTP Requestor as by ...
1. Go to your HTTP requestor's body configuration inside Studio: User-added image · 2. Click on the fx icon and paste the following:...
Read more >
Website response from Nginx empty over TLS in firefox, but ...
... in chrome and safari, however it does not work in firefox. I get an empty response body from the request. It does...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found