Port stripped from request when behind a reverse proxy

I’ve been working on a custom implementation of a reverse proxy filter based on Wildfly with quite a success. The idea behind the proxy is quite simple, provide a basic credential system and re-routing based on request subdomain, so that the same port on the WAN could be used for different local machines, nothing to fancy. This have been working perfectly fine with all web services I’ve tested except with Eclipse Che.

For example for the following super simple configuration:

NAT: 
<wan-ip>:8080 forwarded to <proxy-ip>:8080

Proxy: 
<proxy-ip>:8080 forwarded to <che-server-ip>:8080

With the following address:
proxy: 192.168.0.110
che-server: 192.168.0.111

The dashboard loads fine, but when I try to open a workspace angular throws:

Blocked loading resource from url not allowed by $sceDelegate policy.  URL: http://192.168.0.110/che/wksp-opo2?uid=427793

The problem is as you can see for some reason port gets stripped of the url but only for workspaces. While in dashboard things works fine, in fact if I open the same workspace url that fails adding the port manually it also works fine.

WORKS: http://192.168.0.110:8080/che/wksp-opo2?uid=427793

I really doubt it has something to do with sceDelegate policy itself.

This kind of setup should be transparent to the services running behind the reverse proxy I’ve tested the proxy with other complex services even with websockets and also tested the setup bypassing the proxy forwarding the WAN directly to Che to discard any docker port issue and it worked fine.

Furthermore currently i’m using the proxy on port 80 so that when port are stripped, the request will still be valid urls since default port on http is 80 and its quite nice very similar to what people are trying to achieve with Keycloak but being forced to use port 80 is not fun and quite limiting.

So I was really wondering how or what could be causing this magical ports strip issue. My wild guess is that Che behind the proxy is missing some kind of information and handles workspace in a different way that if it had that information or maybe angular needs some sort of header perhaps any clues of what could be going on?

[TylerJewell]

This has been one of the most educational issues of the year. Reading this thread has been like getting a master class on the nuances of how reverse proxies are supposed to work. Congratulations to @josevimlet on perseverance and a successful implementation.

@eivantsov - do you think there is any value in packaging up the instructions at the end here as a short doc page, or maybe just a short tutorial page?

[lethevimlet]

Good news, I finally managed to get it working behind the Wildfly as a custom reverse proxy with authentication and load balancing.

First, thanks a lot to @eivantsov and @benoitf for your help I really appreciate it, I believe you’ll find pleasing someone using a Redhat server for this purpose ❤️ .

I was on the right track with the the wildcard thing, after studying the Nginx API and @benoitf example, I did my best to port it to Wildfly, and finally I managed to get things working…

The problem was with the fix I used for the port strip issue, rewriteHostHeaders = true, that was messing with the Host header, that’s needed when wildcards (nip.io) where used. I did a nasty quick fix for it with boolean rewriteHostHeader = !exchange.getHostName().contains("nip.io"); to do the rewrite only on the cases that arent nip.io, but I’m planning to tune a lot this approach to create a better solution.

Here are the steps to solve the problems I encountered for using Eclipse Che behind a Wildfly reverse proxy, thou I imagine most of this steps apply to other servers too, hopefully someone will find this useful.

Poxy setup:

1. Create a reverse proxy, in my case I used Undertow LoadBalancingProxyClient and ProxyHandler in a custom handler module for undertow that I wrote. (Note that ProxyHandler parameter rewriteHostHeader fixes port strip issue but mess wildcards used by Che single port mode, so I used the approach described above)

Here is the main code:

        // Proxy the request
        boolean rewriteHostHeader = !exchange.getHostName().contains("nip.io");
        boolean reuseXForwarded = false;
        final ProxyClient proxyClientProvider = new LoadBalancingProxyClient().addHost(URI.create("<CHE_IP_AND_PORT_HERE>"));        
        final HttpHandler handler = new ProxyHandler(proxyClientProvider, 30000, ResponseCodeHandler.HANDLE_404, rewriteHostHeader, reuseXForwarded);;
        
        exchange.dispatch(handler);

1. Added the following headers: X-Forwarded-For: <client_ip> Host: <request_host> X_Forwarded_Proto: http

and the typical CORS headers to allow all (Optional, but just in case)

1. Removed X-Frame-Options header (Once again, just in case)

Eclipse Che setup:

1. Enabled single port mode
2. Set che docker external IP

here is the command to do it directly through the command line although you could also edit che.env

docker run -it —rm -e CHE_SINGLE_PORT=true -e CHE_DOCKER_IP_EXTERNAL=<YOUR_WAN_IP_OR_DNS> -v /var/run/docker.sock:/var/run/docker.sock -v /home/che:/data  eclipse/che start

NAT:

1. Forward your WAN IP to your proxy LAN IP on the same port as CHE_PORT, not sure why but I had issues with nip.io when using a different port (I’ll look at this asap).

Notes:

1. When using CHE_SINGLE_PORT, Che diagnostics will fail.
2. When implementing some sort of authentication bare in mind that request from nip.io come from a different domain and things like session or cookie based authentication will fail. I currently bypass this issue white-listing the authenticated IP + User-Agent thou this solution has its own drawbacks, a better alternative would be to use your own wildcard on the same domain so cookies could propagate in ajax calls, but i’m afraid this wont work unless XMLHttpRequest.withCredentials is set to true on all client XMLHttpRequest, a PR for this would be nice.

And that’s all, hope this help someone to get things going ^^