The label {0} is not a valid A-label'.format(label) - Codepoint U+005F

`2017-07-04 16:19:44 [scrapy.core.scraper] ERROR: Error downloading <GET https://sloneczne_stablowice.forumoteka.pl/kategoria,4,mieszkancy-luzne-rozmowy.html> Traceback (most recent call last): File “c:\users\bukowa\vritualenv2\lib\site-packages\idna\core.py”, line 263, in alabel ulabel(label) File “c:\users\bukowa\vritualenv2\lib\site-packages\idna\core.py”, line 299, in ulabel check_label(label) File “c:\users\bukowa\vritualenv2\lib\site-packages\idna\core.py”, line 253, in check_label raise InvalidCodepoint(‘Codepoint {0} at position {1} of {2} not allowed’.format(_unot(cp_value), pos+1, repr(label))) idna.core.InvalidCodepoint: Codepoint U+005F at position 10 of ‘sloneczne_stablowice’ not allowed

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File “c:\users\bukowa\vritualenv2\lib\site-packages\twisted\internet\defer.py”, line 1384, in inlineCallbacks result = result.throwExceptionIntoGenerator(g) File “c:\users\bukowa\vritualenv2\lib\site-packages\twisted\python\failure.py”, line 393, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb) File “c:\users\bukowa\vritualenv2\lib\site-packages\scrapy\core\downloader\middleware.py”, line 43, in process_request defer.returnValue((yield download_func(request=request,spider=spider))) File “c:\users\bukowa\vritualenv2\lib\site-packages\scrapy\utils\defer.py”, line 45, in mustbe_deferred result = f(*args, **kw) File "c:\users\bukowa\vritualenv2\lib\site-packages\scrapy\core\downloader\handlers_init.py", line 65, in download_request return handler.download_request(request, spider) File “c:\users\bukowa\vritualenv2\lib\site-packages\scrapy\core\downloader\handlers\http11.py”, line 63, in download_request return agent.download_request(request) File “c:\users\bukowa\vritualenv2\lib\site-packages\scrapy\core\downloader\handlers\http11.py”, line 300, in download_request method, to_bytes(url, encoding=‘ascii’), headers, bodyproducer) File “c:\users\bukowa\vritualenv2\lib\site-packages\twisted\web\client.py”, line 1633, in request endpoint = self._getEndpoint(parsedURI) File “c:\users\bukowa\vritualenv2\lib\site-packages\twisted\web\client.py”, line 1617, in _getEndpoint return self._endpointFactory.endpointForURI(uri) File “c:\users\bukowa\vritualenv2\lib\site-packages\twisted\web\client.py”, line 1494, in endpointForURI uri.port) File “c:\users\bukowa\vritualenv2\lib\site-packages\scrapy\core\downloader\contextfactory.py”, line 59, in creatorForNetloc return ScrapyClientTLSOptions(hostname.decode(“ascii”), self.getContext()) File “c:\users\bukowa\vritualenv2\lib\site-packages\twisted\internet_sslverify.py”, line 1152, in init self._hostnameBytes = _idnaBytes(hostname) File “c:\users\bukowa\vritualenv2\lib\site-packages\twisted\internet_idna.py”, line 30, in _idnaBytes return idna.encode(text) File “c:\users\bukowa\vritualenv2\lib\site-packages\idna\core.py”, line 355, in encode result.append(alabel(label)) File “c:\users\bukowa\vritualenv2\lib\site-packages\idna\core.py”, line 265, in alabel raise IDNAError(‘The label {0} is not a valid A-label’.format(label)) idna.core.IDNAError: The label b’sloneczne_stablowice’ is not a valid A-label`

idna == 2.5

[nanvel]

A workaround, patch valid code ranges:

import idna


idna.idnadata.codepoint_classes['PVALID'] = tuple(
    sorted(list(idna.idnadata.codepoint_classes['PVALID']) + [0x5f0000005f])
)

This should not be the problem solution, I agree there should not be undersocres in domain names. But I got an address has underscores, and this is a simple solution to make it works.

[horkhe]

I think you are wrong. Underscores are allowed in domain names, they are not allowed in hostnames which is not the same thing. This explains it: https://stackoverflow.com/questions/2180465/can-domain-name-subdomains-have-an-underscore-in-it.