Mac OS instructions for transparrent proxy don't work

Steps to reproduce the problem:

1. Follow the steps from https://docs.mitmproxy.org/stable/howto-transparent/

Issue1. According to the instructions, editing the sudoers file will allow any user to run /sbin/pfctl -s state

however running this gives the error pfctl: /dev/pf: Permission denied

Issue2. Possible error when running sudo pfctl -f pf.conf

pfctl: Use of -f option, could result in flushing of rules
present in the main ruleset added by the system at startup.
See /etc/pf.conf for further details.

No ALTQ support in kernel
ALTQ related functions disabled

Any other comments? What have you tried so far?

I’m running on mac os 10.14.5, have also tried 10.13 Have confirmed that the sudoers file has been modified and have tried reboot

System information

Mitmproxy: 4.0.4
Python:    3.7.3
OpenSSL:   OpenSSL 1.0.2r  26 Feb 2019
Platform:  Darwin-18.6.0-x86_64-i386-64bit

[sean-freeman]

After reading a lot of docs and old discussions, I have the same issue and will track it here that the docs are incorrect.

The following is done on macOS 10.5 Catalina with:

• Firewall turned on
• WiFi DNS set to 1.1.1.1 instead of Router/ISP default

---

I was receiving the error:

502 Bad Gateway
ProtocolException("Transparent mode failure: RuntimeError('Could not resolve original destination.')")

Until I added into pf (as per @joeltaylor https://github.com/mitmproxy/mitmproxy/issues/2597#issuecomment-430769947):

rdr pass on en0 inet proto tcp from any to any port 8080 -> 127.0.0.1 port 8080

---

On Android mobile device, on adding as a gateway with static ip (as designed):

1. I can see requests to google.com for Play Store etc, but each returns with HTTP 204 because they are rejected by no Certificate.

Client handshake failed. The client may not trust the proxy's certificate for xyz.xyz.com

1. Then after waiting 90 seconds, on browsing to http://mitm.it I see the “Click to install your mitmproxy certificate” page (which must be opened in Chrome to prompt the mitmproxy-ca-cert.pem file download).
2. Most HTTP Requests are then visible, and the phone is working.

On Android mobile device, on adding as a proxy (against guidance of Transparent Proxy):

1. I can see requests to google.com for Play Store etc, but each returns with HTTP 204 because they are rejected by no Certificate.

Client handshake failed. The client may not trust the proxy's certificate for xyz.xyz.com

1. Then after waiting 90 seconds, on browsing to http://mitm.it I see the following error (and for every other URL also):

400 Bad Request
HttpExecption('\n Mitmproxy received an absolute-form request even through it is not running\n in a regular mode. This usually indicates  a misconfiguration,\n please see the mitmproxy mode documentation for details.\n')

---

For ease, I’ve written an on/off switch for .bash_profile which executes the described steps. I have not made this work with localhost yet (ironically, that is what I actually need).

function enable_mitmproxy_transparent (){
    echo ""
    echo "Amending IP Forwarding"
    sudo sysctl -w net.inet.ip.forwarding=1
    sudo sysctl -w net.inet6.ip6.forwarding=1
    
    echo ""
    echo "Checking IP Forwarding"
    sudo sysctl -a | grep forwarding
    
    echo ""
    echo "PF - prepare macOS Packet Filter"
    sudo cp /etc/pf.conf /etc/pf.conf.temp

    # rdr rules in the pf.conf given above only apply to inbound traffic
    # will NOT redirect traffic coming from the box running pf itself.
    # sed commands are ugly without identation due to how sed handles newline on macOS
    sudo sed -i .bak '/rdr-anchor/a\
rdr pass on en0 inet proto tcp from any to any port 80 -> 127.0.0.1 port 8080
' /etc/pf.conf.temp
    sudo sed -i .bak '/rdr pass/a\
rdr pass on en0 inet proto tcp from any to any port 443 -> 127.0.0.1 port 8080
' /etc/pf.conf.temp
    sudo sed -i .bak '/rdr pass/a\
rdr pass on en0 inet proto tcp from any to any port 8080 -> 127.0.0.1 port 8080
' /etc/pf.conf.temp

    echo ""
    echo "PF - enable with new configuration file"
    sudo pfctl -ef /etc/pf.conf.temp
    sudo pfctl -E

    echo ""
    echo "PF - confirm Translation (i.e. BAT) rules are loaded"
    sudo pfctl -s nat

    echo ""
    echo "PF - Check to see connections from the state table"
    sudo pfctl -s states

    #echo ""
    #echo "PF - DEBUG Check all filters etc." 
    #sudo pfctl -s all

    echo ""
    echo "Amend sudoers to allow all macOS Admins to see PF State Table using pfctl -s state"
    sudo cp /etc/sudoers /etc/sudoers.backup
    sudo bash -c 'echo "%admin ALL=(ALL:ALL) NOPASSWD: /sbin/pfctl -s state" >> /etc/sudoers' 

    echo ""
    echo "Ready to run mitmproxy proxy machine in Transparent Proxy Mode, with Client configured as custom gateway."
    echo ""
    echo "Source 1: https://docs.mitmproxy.org/stable/concepts-modes/#transparent-proxy"
    echo "Source 2: https://docs.mitmproxy.org/stable/howto-transparent/"
    echo "Source 3: https://docs.mitmproxy.org/stable/concepts-certificates/"
    echo ""
    echo ""
    echo "Short Instructions:"

    echo "1. Run mitmproxy as normal (not able to monitor macOS local network traffic)..."
    find_ip=$(ifconfig en0 | grep 'inet' | grep -v 'inet6' | awk '{print $2}')
    echo "sudo mitmproxy --mode transparent --showhost"

    echo ""
    echo "2. Go to Mobile Device, edit WiFi settings. Change IP Settings to Static."

    echo ""
    echo "3. Choose a static IP, and add ** $find_ip ** as the Default Gateway."

    echo ""
    echo "4. Open a browser to http://mitm.it on the Mobile Device, add Certificate."

    echo ""

    # Run if monitoring local traffic
    #echo "Now run....."
    #echo "sudo -u nobody mitmproxy --mode transparent --showhost"
}

function disable_mitmproxy_transparent (){
    sudo sysctl -w net.inet.ip.forwarding=0
    sudo sysctl -w net.inet6.ip6.forwarding=0
    sudo pfctl -ef /etc/pf.conf
    sudo rm /etc/pf.conf.temp
    sudo rm /etc/pf.conf.temp.bak
    sudo mv /etc/sudoers.backup /etc/sudoers
}

export -f enable_mitmproxy_transparent
export -f disable_mitmproxy_transparent

[ConfusedVorlon]

update - opening localhost:8080 on the machine running the proxy gives the response

502 Bad Gateway
ProtocolException("Transparent mode failure: RuntimeError('Could not resolve original destination.')")

this looks much like https://github.com/mitmproxy/mitmproxy/issues/2597 although editing rootcontext.py doesn’t help me