webtest.openURL sends an absolute URI when passed as "url"

• I’m submitting a … [x] bug report [ ] feature request [ ] question about the decisions made in the repository

• Do you want to request a feature or report a bug?

Currently openURL will pass the provided URL directly to the remote server, without parsing it. This no longer works due to changes made in cherrypy/cheroot#39 that disallows absolute URI’s unless the server is configured as a proxy server.

• What is the current behavior?

💥

• If the current behavior is a bug, please provide the steps to reproduce and if possible a screenshots and logs of the problem.

[2017-08-15 21:11:17,794] (/home/vagrant/src/backend-http/tests/controllers/base.py:base:GET) - DEBUG - Requesting URL: http://localhost:35100/api/account/users/nothere/
Absolute URI not allowed if server is not a proxy.

• What is the expected behavior?

That openURL does the appropriate thing and sends a valid HTTP request.

• What is the motivation / use case for changing the behavior?

The RFC’s state that absolute URI’s are not allowed to be sent to a server unless it is a proxy server.

• Other information

This is my work-around in a pytest fixture, this works for me. Most likely openURL should update it’s host/port information if receiving an absolute URI to be able to connect to alternate host/port than what is configured for the WebCase.

    from cherrypy.test.webtest import openURL
    from urlparse import urlsplit
    _broken_openURL = openURL

    def openURL(url, *args, **kw):
        scheme, authority, path, qs, fragment = urlsplit(url)
        return _broken_openURL(path + '?' + qs, *args, **kw)

    webtest.openURL = openURL

[bertjwregeer]

No, this doesn’t satisfy my needs. All of our testing uses webtest.openURL which does the wrong thing, it sends the full URL as part of the request:

GET http://fullurl.com:port/is/here/ HTTP/1.1

The above is an absoluteURI request. The new openAbsoluteURI you just created actually turns the request into an abs_path request, which is the only thing that is valid to send to a non-proxy server.

See https://www.w3.org/Protocols/rfc2616/rfc2616-sec5.html section 5.1.2:

The absoluteURI form is only allowed when the request is being made to a proxy.

Since in almost all cases of testing, when the user passes a URL to openURL they want to do a request against the locally running cherrypy you should NOT pass an absoluteURI instead you need to parse it, and send an abs_path with the query string parameters.

[webknjaz]

I’ve added cheroot.test.webtest.openAbsoluteURI. Hopefully it will satisfy your needs.