Allow blacklisting HTTPS requests without connecting to server first
See original GitHub issueThis is a problem because these sorts of pathological situations are exactly why I started using browsermob-proxy. Workaround is to use AdvancedHostResolver#remapHost
instead, but this is unsightly and might fail in some corner cases.
I think it probably has something to do with the order the HttpFiltersSourceAdapter
s are applied in BrowerMobProxyServer
, but I’m new to this code base. I think either ResolvedHostnameCacheFilter
or HttpsOriginalHostCaptureFilter
requires a valid SSL socket to be created before the blacklist is applied in BlacklistFilter
.
Using bmp 2.1.1 and selenium 2.53.1.
Test case:
@Test
public void bugReport() throws Exception {
final BrowserMobProxyServer server = new BrowserMobProxyServer();
server.blacklistRequests(".*google\\.com.*", 204);
server.blacklistRequests(".*licdn\\.com.*", 204);
server.start();
final Proxy proxy = ClientUtil.createSeleniumProxy(server);
final DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability(PROXY, proxy);
final FirefoxDriver driver = new FirefoxDriver(capabilities);
// works fine
driver.get("https://www.google.com/");
// java.net.SocketException
driver.get("https://media-vnsh.licdn.com/");
}
DEBUG logs:
2016-07-21 17:04:32,868 [:main] DEBUG ImpersonatingMitmManager - Allowed ciphers for proxy connections to upstream servers (some ciphers may not be available): [TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, TLS_RSA_WITH_AES_256_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384, TLS_DHE_RSA_WITH_AES_256_CBC_SHA256, TLS_DHE_DSS_WITH_AES_256_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDH_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_DSS_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384, TLS_DHE_RSA_WITH_AES_256_GCM_SHA384, TLS_DHE_DSS_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_ECDSA_WITH_RC4_128_SHA, TLS_ECDHE_RSA_WITH_RC4_128_SHA, SSL_RSA_WITH_RC4_128_SHA, TLS_ECDH_ECDSA_WITH_RC4_128_SHA, TLS_ECDH_RSA_WITH_RC4_128_SHA, SSL_RSA_WITH_RC4_128_MD5, TLS_EMPTY_RENEGOTIATION_INFO_SCSV]
2016-07-21 17:04:32,869 [:main] DEBUG ImpersonatingMitmManager - Allowed ciphers for client connections to proxy (some ciphers may not be available): [TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, TLS_RSA_WITH_AES_256_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384, TLS_DHE_RSA_WITH_AES_256_CBC_SHA256, TLS_DHE_DSS_WITH_AES_256_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDH_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_DSS_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384, TLS_DHE_RSA_WITH_AES_256_GCM_SHA384, TLS_DHE_DSS_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_ECDSA_WITH_RC4_128_SHA, TLS_ECDHE_RSA_WITH_RC4_128_SHA, SSL_RSA_WITH_RC4_128_SHA, TLS_ECDH_ECDSA_WITH_RC4_128_SHA, TLS_ECDH_RSA_WITH_RC4_128_SHA, SSL_RSA_WITH_RC4_128_MD5, TLS_EMPTY_RENEGOTIATION_INFO_SCSV]
2016-07-21 17:04:32,990 [:main] INFO DefaultHttpProxyServer - Starting proxy at address: 0.0.0.0/0.0.0.0:0
2016-07-21 17:04:33,049 [:main] INFO DefaultHttpProxyServer - Proxy listening with TCP transport
2016-07-21 17:04:33,454 [:main] INFO DefaultHttpProxyServer - Proxy started at address: /0:0:0:0:0:0:0:0:37211
2016-07-21 17:04:36,264 [:LittleProxy-0-ClientToProxyWorker-1] DEBUG ImpersonatingMitmManager - Impersonated certificate for tiles-cloudfront.cdn.mozilla.net in 462ms
2016-07-21 17:04:36,349 [:LittleProxy-0-ClientToProxyWorker-0] DEBUG ImpersonatingMitmManager - Impersonated certificate for www.google.com in 547ms
2016-07-21 17:04:57,382 [:LittleProxy-0-ProxyToServerWorker-2] INFO ProxyToServerConnection - (CONNECTING) [id: 0x6af86dfa]: Connection to upstream server failed
java.net.SocketException: Socket is not connected: no further information: media-vnsh.licdn.com/197.96.19.88:443
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:224)
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:289)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:545)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:485)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:399)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:371)
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
at java.lang.Thread.run(Thread.java:745)
2016-07-21 17:04:57,444 [:LittleProxy-JVM-shutdown-hook] INFO DefaultHttpProxyServer - Shutting down proxy server immediately (non-graceful)
2016-07-21 17:04:57,444 [:LittleProxy-JVM-shutdown-hook] INFO DefaultHttpProxyServer - Closing all channels (non-graceful)
2016-07-21 17:04:57,450 [:LittleProxy-JVM-shutdown-hook] INFO ServerGroup - Shutting down server group event loops (non-graceful)
2016-07-21 17:04:57,452 [:LittleProxy-JVM-shutdown-hook] INFO DefaultHttpProxyServer - Done shutting down proxy server
Issue Analytics
- State:
- Created 7 years ago
- Reactions:1
- Comments:8
Top Results From Across the Web
A way to whitelist (or blacklist) https connections - Server Fault
While blocking and allowing http requests is easy: WifiDog has an option to redirect http to a transparent proxy. I whitelisted the required ......
Read more >Blacklisting vs. Whitelisting | 2021 Guide | Consoildated Tech.
The blacklist approach is threat-centric, and the default is to allow access. Any entity not on the blacklist is granted access, but anything...
Read more >Eight Ways to Block and Redirect with Apache's mod_rewrite
Blacklist via Request Method. [ #1 ] This first blacklisting method evaluates the client's request method. Every time a client attempts to ...
Read more >Cross-Origin Resource Sharing (CORS) - MDN Web Docs
The request header of note is Origin , which shows that the invocation is coming from https://foo.example . Now let's see how the...
Read more >Server Side Request Forgery (SSRF) Attacks & How to ...
Allow only URL schemas that your application uses. There is no need to have ftp://, file:/// or even http:// enabled if you only...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Not yet, unfortunately. I’m on the verge of releasing the next version of the REST API for comment. ConnectionFlow refactor is the next big thing on my list though.
Any updates on this issue? I’m having problems with a page that takes so long to load and I want to blacklist it, but the page is under HTTPS and is still being loaded.
And AdvancedHostResolver is not an option, since I still need to open the HTTPS website.