server <ip> down / VIRTUAL_PORT ignored

Hi,

how would I go about debugging the container discovery?

I run a gateone server (docker-compose below) and it is no longer recognized by nginx-proxy. It happened after the last nginx-proxy update, but I’m not sure if it is really related to the update.

Basically the server listens on 8701, the docker-compose has an expose: 8701 and a VIRTUAL_PORT: 8701 but the default.conf has server <ip> down;. Otherwise the default.conf is entirely correct, ip and everything. And if I change the default.conf to server <ip>:8701; everything works. But there is no error message in the log and I don’t know enough about the way nginx-proxy builds the default.conf to help you with a specific reason.

Thanks, Stefan

version: '2'

networks:
  proxy-tier:
    external:
      name: external

services:
  gateone-server:
    build: ./gateone_statup
    image: stefanfritsch/gateone_statup
    restart: unless-stopped
    container_name: gateone-server
    expose:
    - 8701
    networks:
    - proxy-tier
    environment:
      VIRTUAL_HOST: gateone.example.com
      VIRTUAL_PORT: 8701
      VIRTUAL_NETWORK: proxy-tier
      LETSENCRYPT_HOST: gateone.example.com
      LETSENCRYPT_EMAIL: [e-mail]

  gateone-auth:
    image: zappi/oauth2_proxy
    restart: unless-stopped
    container_name: gateone-auth
    networks:
    - proxy-tier
    depends_on:
      - gateone-server
    command: --cookie-secure=true --provider="azure" --upstream="http://gateone-server:8701" --http-address="0.0.0.0:4180" --redirect-url="https://gateone.example.com/oauth2/callback" --email-domain="example.com"
    environment: ...

Versions

root@se-dino:/mnt/Container Data/ssl/nginx-proxy# docker version
Client:
 Version:      17.04.0-ce
 API version:  1.28
 Go version:   go1.7.5
 Git commit:   4845c56
 Built:        Mon Apr  3 18:07:42 2017
 OS/Arch:      linux/amd64

Server:
 Version:      17.04.0-ce
 API version:  1.28 (minimum version 1.12)
 Go version:   go1.7.5
 Git commit:   4845c56
 Built:        Mon Apr  3 18:07:42 2017
 OS/Arch:      linux/amd64
 Experimental: false
root@se-dino:/mnt/Container Data/ssl/nginx-proxy# docker-compose version  # if you are using docker-compose
docker-compose version 1.11.2, build dfed245
docker-py version: 2.1.0
CPython version: 2.7.13
OpenSSL version: OpenSSL 1.0.1t  3 May 2016
root@se-dino:/mnt/Container Data/ssl/nginx-proxy# docker images --format "{{.ID}}\t{{.Repository}}:{{.Tag}}" jwilder/nginx-proxy
f2e03275b4ca    jwilder/nginx-proxy:latest

Log

root@se-dino:/mnt/Container Data/ssl/nginx-proxy# docker-compose logs nginx-proxy
Attaching to nginx-proxy
nginx-proxy          | forego     | starting dockergen.1 on port 5000
nginx-proxy          | forego     | starting nginx.1 on port 5100
nginx-proxy          | dockergen.1 | 2017/04/27 09:16:50 Generated '/etc/nginx/conf.d/default.conf' from 13 containers
nginx-proxy          | dockergen.1 | 2017/04/27 09:16:50 Running 'nginx -s reload'
nginx-proxy          | dockergen.1 | 2017/04/27 09:16:50 Watching docker events
nginx-proxy          | dockergen.1 | 2017/04/27 09:16:50 Contents of /etc/nginx/conf.d/default.conf did not change. Skipping notification 'nginx -s reload'
nginx-proxy          | nginx.1    | 2017/04/27 09:16:54 [error] 75#75: *1 no live upstreams while connecting to upstream, client: 82.135.86.221, server: gateone.example.com, request: "GET /ws HTTP/1.1", upstream: "http://gateone.example.com/ws", host: "gateone.example.com"
nginx-proxy          | nginx.1    | gateone.example.com 82.135.86.221 - fritsch [27/Apr/2017:09:16:54 +0000] "GET /ws HTTP/1.1" 502 576 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36"

default.conf

root@se-dino:/mnt/Container Data/ssl/nginx-proxy# docker exec nginx-proxy cat /etc/nginx/conf.d/default.conf
# If we receive X-Forwarded-Proto, pass it through; otherwise, pass along the
# scheme used to connect to this server
map $http_x_forwarded_proto $proxy_x_forwarded_proto {
  default $http_x_forwarded_proto;
  ''      $scheme;
}
# If we receive X-Forwarded-Port, pass it through; otherwise, pass along the
# server port the client connected to
map $http_x_forwarded_port $proxy_x_forwarded_port {
  default $http_x_forwarded_port;
  ''      $server_port;
}
# If we receive Upgrade, set Connection to "upgrade"; otherwise, delete any
# Connection header that may have been passed to this server
map $http_upgrade $proxy_connection {
  default upgrade;
  '' close;
}
# Set appropriate X-Forwarded-Ssl header
map $scheme $proxy_x_forwarded_ssl {
  default off;
  https on;
}
gzip_types text/plain text/css application/javascript application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;
log_format vhost '$host $remote_addr - $remote_user [$time_local] '
                 '"$request" $status $body_bytes_sent '
                 '"$http_referer" "$http_user_agent"';
access_log off;
# HTTP 1.1 support
proxy_http_version 1.1;
proxy_buffering off;
proxy_set_header Host $http_host;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $proxy_connection;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $proxy_x_forwarded_proto;
proxy_set_header X-Forwarded-Ssl $proxy_x_forwarded_ssl;
proxy_set_header X-Forwarded-Port $proxy_x_forwarded_port;
# Mitigate httpoxy attack (see README for details)
proxy_set_header Proxy "";
server {
        server_name _; # This is just an invalid value which will never trigger on a real hostname.
        listen 80;
        access_log /var/log/nginx/access.log vhost;
        return 503;
}
# gateone.example.com
upstream gateone.example.com {
                                ## Can be connect with "external" network
                # gateone_gateone-server_1
                server 172.20.0.3 down;
}
server {
        server_name gateone.example.com;
        listen 80 ;
        access_log /var/log/nginx/access.log vhost;
        return 301 https://$host$request_uri;
}
server {
        server_name gateone.example.com;
        listen 443 ssl http2 ;
        access_log /var/log/nginx/access.log vhost;
        ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
        ssl_ciphers '...';
        ssl_prefer_server_ciphers on;
        ssl_session_timeout 5m;
        ssl_session_cache shared:SSL:50m;
        ssl_session_tickets off;
        ssl_certificate /etc/nginx/certs/gateone.example.com.crt;
        ssl_certificate_key /etc/nginx/certs/gateone.example.com.key;
        ssl_dhparam /etc/nginx/certs/gateone.example.com.dhparam.pem;
        add_header Strict-Transport-Security "max-age=31536000";
        include /etc/nginx/vhost.d/gateone.example.com;
        location / {
                proxy_pass http://gateone.example.com;
                include /etc/nginx/vhost.d/gateone.example.com_location;
        }
}

gateone.example.com

## Start of configuration add by letsencrypt container
location ^~ /.well-known/acme-challenge/ {
    allow all;
    root /usr/share/nginx/html;
    try_files $uri =404;
    break;
}
## End of configuration add by letsencrypt container
  location = /oauth2/auth {
    internal;
    proxy_pass http://gateone-auth:4180;
  }

  location /oauth2/ {
    proxy_pass http://gateone-auth:4180;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Scheme $scheme;
  }

gateone.example.com_location

auth_request /oauth2/auth;
error_page 401 = https://gateone.example.com/oauth2/sign_in;
proxy_buffering off;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";

[smoebody]

you can explicitly start the container with the port exposed that you want to connect to, e.g.:

docker run --expose 1234 -e 'VIRTUAL_PORT=1234' -e 'VIRTUAL_HOST=example.com" my/image

[titpetric]

@kaminek Adding something like EXPOSE 80 into your Dockerfile seems to solve the issue.