Type hints and Response.selector

Python 3.5 introduced Type Hints: https://docs.python.org/3/library/typing.html

I’m using them with Scrapy’s response object, but it seems that however you’ve declared selectors in scrapy it makes PyCharm unable to realise there’s a selector method/attribute:

from scrapy.http import Response

def my_function(response: Response):
    response.selector.remove_namespaces()

For which PyCharm reports: “Unresolved attribute reference ‘selector’ for class ‘Response’” Obviously the selector works, but it’s not being exposed sufficiently well for the IDE to be able to spot it which obviates the point of type hints.

[kmike]

probably you need

def my_function(response: Union[Response, TextResponse]):
    assert isinatance(response, TextResponse)
    response.selector.remove_namespaces()

or just

from scrapy.http import TextResponse

def my_function(response: TextResponse):
    response.selector.remove_namespaces()

[kmike]

Type hints are correct in pointing this out - Response doesn’t have .selector attribute, only TextResponse does. Scrapy selects Response subclass based on server response body and headers; for binary data it’d be just Response (=> no parsed HTML tree), while for HTML it’d be HtmlResponse, which is a Response subclass.