After the python3 fix for CVE-2019-9740 test_null_bytes fails

I’m submitting a …

• bug report
• feature request
• question about the decisions made in the repository

Do you want to request a feature or report a bug? bug

What is the current behavior? When running the testsuite StaticTest.test_null_bytes fails with:

[  107s] __________________________ StaticTest.test_null_bytes __________________________
[  107s] 
[  107s] self = <cherrypy.test.test_static.StaticTest testMethod=test_null_bytes>
[  107s] 
[  107s]     def test_null_bytes(self):
[  107s] >       self.getPage('/static/\x00')
[  107s] 
[  107s] cherrypy/test/test_static.py:401: 
[  107s] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[  107s] cherrypy/test/helper.py:326: in getPage
[  107s]     protocol, raise_subcls)
[  107s] /usr/lib/python3.7/site-packages/cheroot/test/webtest.py:212: in getPage
[  107s]     raise_subcls, ssl_context=self.ssl_context,
[  107s] /usr/lib/python3.7/site-packages/cheroot/test/webtest.py:514: in openURL
[  107s]     skip_accept_encoding=True,
[  107s] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[  107s] 
[  107s] self = <http.client.HTTPConnection object at 0x7f1697a032b0>, method = 'GET'
[  107s] url = '/static/\x00', skip_host = True, skip_accept_encoding = True
[  107s] 
[  107s]     def putrequest(self, method, url, skip_host=False,
[  107s]                    skip_accept_encoding=False):
[  107s]         """Send a request to the server.
[  107s]     
[  107s]         `method' specifies an HTTP request method, e.g. 'GET'.
[  107s]         `url' specifies the object being requested, e.g. '/index.html'.
[  107s]         `skip_host' if True does not add automatically a 'Host:' header
[  107s]         `skip_accept_encoding' if True does not add automatically an
[  107s]            'Accept-Encoding:' header
[  107s]         """
[  107s]     
[  107s]         # if a prior response has been completed, then forget about it.
[  107s]         if self.__response and self.__response.isclosed():
[  107s]             self.__response = None
[  107s]     
[  107s]     
[  107s]         # in certain cases, we cannot issue another request on this connection.
[  107s]         # this occurs when:
[  107s]         #   1) we are in the process of sending a request.   (_CS_REQ_STARTED)
[  107s]         #   2) a response to a previous request has signalled that it is going
[  107s]         #      to close the connection upon completion.
[  107s]         #   3) the headers for the previous response have not been read, thus
[  107s]         #      we cannot determine whether point (2) is true.   (_CS_REQ_SENT)
[  107s]         #
[  107s]         # if there is no prior response, then we can request at will.
[  107s]         #
[  107s]         # if point (2) is true, then we will have passed the socket to the
[  107s]         # response (effectively meaning, "there is no prior response"), and
[  107s]         # will open a new one when a new request is made.
[  107s]         #
[  107s]         # Note: if a prior response exists, then we *can* start a new request.
[  107s]         #       We are not allowed to begin fetching the response to this new
[  107s]         #       request, however, until that prior response is complete.
[  107s]         #
[  107s]         if self.__state == _CS_IDLE:
[  107s]             self.__state = _CS_REQ_STARTED
[  107s]         else:
[  107s]             raise CannotSendRequest(self.__state)
[  107s]     
[  107s]         # Save the method we use, we need it later in the response phase
[  107s]         self._method = method
[  107s]         if not url:
[  107s]             url = '/'
[  107s]         # Prevent CVE-2019-9740.
[  107s]         if _contains_disallowed_url_pchar_re.search(url):
[  107s] >           raise InvalidURL(f"URL can't contain control characters. {url!r}")
[  107s] E           http.client.InvalidURL: URL can't contain control characters. '/static/\x00'
[  107s]

Throwing an exception here is actually completely correct behavior after fixing https://github.com/python/cpython/pull/13044 (CVE-2019-9740).

If the current behavior is a bug, please provide the steps to reproduce and if possible a screenshots and logs of the problem. If you can, show us your code.

Full build log

What is the expected behavior? Whole test suite should pass.

What is the motivation / use case for changing the behavior? Packaging python-CherryPy for openSUSE.

Please tell us about your environment: Linux/openSUSE/Tumbleweed

• Cheroot version: 6.55
• CherryPy version: 18.1.1
• Python version: 3.7.3
• OS: openSUSE/Tumbleweed
• Browser: not in the browser

Other information (e.g. detailed explanation, stacktraces, related issues, suggestions how to fix, links for us to have context, e.g. stackoverflow, gitter, etc.)

[tipabu]

Well, a similar one, anyway 😉 When I’d originally written it, you could still send null bytes just fine, but not b'\x80'through b'\xff'

I’d love to see an upstream patch that accommodates both use-cases, though!

[jaraco]

There is an upstream issue36274 tracking this use-case.