Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Requesting complete URL over https proxy results in "Invalid request scheme: https"

See original GitHub issue

Steps to reproduce the problem:

Set up an HTTPS mitmproxy
Run the following (minimized) code against it:

import httplib
conn = httplib.HTTPSConnection('<your proxy IP>',<your proxy port>)
conn.set_tunnel('www.google.com',443)
conn.request('GET','https://www.google.com:443/')
resp = conn.getresponse()
print resp.read()

What is the expected behavior?

<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>301 Moved</TITLE></HEAD><BODY>
<H1>301 Moved</H1>
The document has moved
<A HREF="http://www.google.com/">here</A>.
</BODY></HTML>

What went wrong?

<html>
            <head>
                <title>400 Bad Request</title>
            </head>
            <body>HttpException('Invalid request scheme: https',)</body>
        </html>

Any other comments?

I believe it’s caused by a combination of https://github.com/mitmproxy/mitmproxy/blob/master/libmproxy/protocol/http.py#L544-L545 and https://github.com/mitmproxy/mitmproxy/blob/master/libmproxy/protocol/http.py#L502-L503. Indeed, if you modify my GET request above to ask for just / instead of https://www.google.com:443/, everything works fine.

I’m not sure if it’s sensible to support absolute-form requests over https tunnels, but I’m getting this behavior in real code (boto for python AWS) so I don’t think it’s very a contrived situation.

mitmproxy version: 0.14.0

Issue Analytics

State:
Created 8 years ago
Reactions:7
Comments:28 (9 by maintainers)

Top GitHub Comments

6reactions

abishekvashokcommented, Jul 11, 2018

--relax-http-form-validation whats the new version of this? This param doesnot work in the newest binary

5reactions

drorwolmercommented, Feb 2, 2021

If someone needs a quick/hacky fix, change mitmproxy/net/http/http1/read.py

    try:
        import logging
        logging.error(line)
        method, path, http_version = line.split()

        import urllib.parse
        logging.error(urllib.parse.urlparse(path))

        if path == b"*" or path.startswith(b"/"):
            form = "relative"
            scheme, host, port = None, None, None
        elif method == b"CONNECT":
            form = "authority"
            host, port = _parse_authority_form(path)
            scheme, path = None, None
        else:
            form = "absolute" . # <<<<< Change this to `relative`
            scheme, host, port, path = url.parse(path)

        _check_http_version(http_version)

IMG_20200309_122625_exported_stabilized_2057254493992227421

Top Results From Across the Web

Why am I seeing InvalidProxyConfigurationWarning when ...

This warning is new in urllib3 v1.25.9 and means that your proxy which is configured to use HTTPS is not doing what you...

Proxy support in Chrome

When proxying https:// requests through an HTTP proxy, the TLS exchange is forwarded through the proxy using the CONNECT method, so end-to-end encryption...

400 Bad request - plain HTTP request sent to HTTPS port

This error occurs if you have incorrectly configured HTTP requests to a TLS-enabled backend server in the target endpoint of an API Proxy....

RFC 3261: SIP: Session Initiation Protocol

SIP makes use of elements called proxy servers to help route requests to the user's current location, authenticate and authorize users for services, ......

Working with SoapUI | Preferences

SSL Settings, Define keystore, and Mock service SSL parameters. ... Chunking Threshold, Using content-chunking for requests larger than thresholds, ...