Kibana with Ingress - Endpoint has no IP

Chart version: 7.2.0

Kubernetes version: 1.13.6-gke.13

Kubernetes provider: GKE

Helm Version: 2.14.1

helm get release output

REVISION: 1
RELEASED: Tue Jul  9 14:54:53 2019
CHART: kibana-7.2.0
USER-SUPPLIED VALUES:
elasticsearchHosts: http://elasticsearch-master:9200
ingress:
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/auth-realm: Monitoring Authentication Required - kibanaadmin
    nginx.ingress.kubernetes.io/auth-secret: monitoring-ingress-auth
    nginx.ingress.kubernetes.io/auth-type: basic
  enabled: true
  hosts:
  - beta-monitoring.xeecloud.io
  path: /hizen/kibana
  tls:
  - hosts:
    - beta-monitoring.xeecloud.io
    secretName: beta-monitoring-xeecloud-io-tls
kibanaConfig:
  kibana.yml: |
    server.basePath: /hizen/kibana
    server.rewriteBasePath: true
service:
  nodePort: 32601
  type: NodePort

COMPUTED VALUES:
affinity: {}
antiAffinity: hard
antiAffinityTopologyKey: kubernetes.io/hostname
elasticsearchHosts: http://elasticsearch-master:9200
elasticsearchURL: ""
extraEnvs: []
fullnameOverride: ""
healthCheckPath: /app/kibana
httpPort: 5601
image: docker.elastic.co/kibana/kibana
imagePullPolicy: IfNotPresent
imagePullSecrets: []
imageTag: 7.2.0
ingress:
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/auth-realm: Monitoring Authentication Required - kibanaadmin
    nginx.ingress.kubernetes.io/auth-secret: monitoring-ingress-auth
    nginx.ingress.kubernetes.io/auth-type: basic
  enabled: true
  hosts:
  - beta-monitoring.xeecloud.io
  path: /hizen/kibana
  tls:
  - hosts:
    - beta-monitoring.xeecloud.io
    secretName: beta-monitoring-xeecloud-io-tls
kibanaConfig:
  kibana.yml: |
    server.basePath: /hizen/kibana
    server.rewriteBasePath: true
maxUnavailable: 1
nameOverride: ""
nodeSelector: {}
podSecurityContext:
  fsGroup: 1000
priorityClassName: ""
protocol: http
readinessProbe:
  failureThreshold: 3
  initialDelaySeconds: 10
  periodSeconds: 10
  successThreshold: 3
  timeoutSeconds: 5
replicas: 1
resources:
  limits:
    cpu: 1000m
    memory: 1Gi
  requests:
    cpu: 100m
    memory: 500m
secretMounts: []
securityContext:
  capabilities:
    drop:
    - ALL
  runAsNonRoot: true
  runAsUser: 1000
serverHost: 0.0.0.0
service:
  annotations: {}
  nodePort: 32601
  port: 5601
  type: NodePort
serviceAccount: ""
tolerations: []
updateStrategy:
  type: Recreate

HOOKS:
MANIFEST:

---
# Source: kibana/templates/configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: kibana-kibana-config
  labels:
    app: kibana
    release: "kibana"
data:
  kibana.yml: |
    server.basePath: /hizen/kibana
    server.rewriteBasePath: true
---
# Source: kibana/templates/service.yaml
apiVersion: v1
kind: Service
metadata:
  name: kibana-kibana
  labels:
    app: kibana
    release: "kibana"
    heritage: Tiller
spec:
  type: NodePort
  ports:
    - port: 5601
      nodePort: 32601
      protocol: TCP
      name: http
      targetPort: 5601
  selector:
    app: kibana
    release: "kibana"
---
# Source: kibana/templates/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: kibana-kibana
  labels:
    app: kibana
    release: "kibana"
spec:
  replicas: 1
  strategy:
    type: Recreate

  selector:
    matchLabels:
      app: kibana
      release: "kibana"
  template:
    metadata:
      labels:
        app: kibana
        release: "kibana"
      annotations:

        configchecksum: dac182bd9bd08905e383049bb219ef778ba0adaddda1c4b57fdb9fc6b94d59d
    spec:
      securityContext:
        fsGroup: 1000

      volumes:
        - name: kibanaconfig
          configMap:
            name: kibana-kibana-config
      containers:
      - name: kibana
        securityContext:
          capabilities:
            drop:
            - ALL
          runAsNonRoot: true
          runAsUser: 1000

        image: "docker.elastic.co/kibana/kibana:7.2.0"
        env:
          - name: ELASTICSEARCH_HOSTS
            value: "http://elasticsearch-master:9200"
          - name: SERVER_HOST
            value: "0.0.0.0"
        readinessProbe:
          failureThreshold: 3
          initialDelaySeconds: 10
          periodSeconds: 10
          successThreshold: 3
          timeoutSeconds: 5

          exec:
            command:
              - sh
              - -c
              - |
                #!/usr/bin/env bash -e
                http () {
                    local path="${1}"
                    set -- -XGET -s --fail

                    if [ -n "${ELASTIC_USERNAME}" ] && [ -n "${ELASTIC_PASSWORD}" ]; then
                      set -- "$@" -u "${ELASTIC_USERNAME}:${ELASTIC_PASSWORD}"
                    fi

                    curl -k "$@" "http://localhost:5601${path}"
                }

                http "/app/kibana"
        ports:
        - containerPort: 5601
        resources:
          limits:
            cpu: 1000m
            memory: 1Gi
          requests:
            cpu: 100m
            memory: 500m

        volumeMounts:
          - name: kibanaconfig
            mountPath: /usr/share/kibana/config/kibana.yml
            subPath: kibana.yml
---
# Source: kibana/templates/ingress.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: kibana-kibana
  labels:
    app: kibana
    release: kibana
    heritage: Tiller
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/auth-realm: Monitoring Authentication Required - kibanaadmin
    nginx.ingress.kubernetes.io/auth-secret: monitoring-ingress-auth
    nginx.ingress.kubernetes.io/auth-type: basic

spec:
  tls:
    - hosts:
      - beta-monitoring.xeecloud.io
      secretName: beta-monitoring-xeecloud-io-tls

  rules:
    - host: beta-monitoring.xeecloud.io
      http:
        paths:
          - path: /hizen/kibana
            backend:
              serviceName: kibana-kibana
              servicePort: 5601

Describe the bug: Trying to add an ingress to my Kibana, I always end up having a 503 in my Ingress because the kibana endpoint does not have IPs.

Steps to reproduce:

1. Deploy the helm chart with an Ingress configured
2. Try to access Kibana through the Ingress
3. Describe the kibana endpoint

Expected behavior:

Endpoint should have an IP and be reachable through the ingress.

Provide logs and/or server output (if relevant):

NGINX Ingress Controller log when calling the endpoint (note that we have - - - - instead of an IP)

10.0.3.1 - [10.0.3.1] - kibanaadmin [09/Jul/2019:13:01:03 +0000] "GET /hizen/kibana HTTP/2.0" 503 600 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36" 146 0.000 [monitoring-kibana-kibana-5601] - - - - b38299d11a01c44678218eb7525521b9

Any additional context: I tried with ClusterIP and NodePort, same behaviour.

values file

elasticsearchHosts: "http://elasticsearch-master:9200"

kibanaConfig:
  kibana.yml: |
    server.basePath: /hizen/kibana
    server.rewriteBasePath: true

service:
  type: NodePort
  nodePort: 32601

ingress:
  enabled: true
  path: /hizen/kibana
  hosts:
    - beta-monitoring.xeecloud.io
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/auth-type: basic
    nginx.ingress.kubernetes.io/auth-secret: monitoring-ingress-auth
    nginx.ingress.kubernetes.io/auth-realm: "Monitoring Authentication Required - kibanaadmin"
  tls:
    - secretName: beta-monitoring-xeecloud-io-tls
      hosts:
        - beta-monitoring.xeecloud.io

describe of the endpoint

Name:         kibana-kibana
Namespace:    monitoring
Labels:       app=kibana
              heritage=Tiller
              release=kibana
Annotations:  <none>
Subsets:
  Addresses:          <none>
  NotReadyAddresses:  10.0.1.83
  Ports:
    Name  Port  Protocol
    ----  ----  --------
    http  5601  TCP

Events:  <none>

Note: I tried setting the server.host to 0.0.0.0 as well as mentionned in https://github.com/elastic/helm-charts/issues/156 but it does not work either and does not seem to be linked to an endpoint.

[Crazybus]

I think the issue here is that the health check is failing because it hasn’t been configured to look at the basePath. If the health check is failing then the pod isn’t added into the service.

Can you try setting:

healthCheckPath: "/hizen/kibana/app/kibana"

I’m also working on a PR to make sure this is mentioned in the readme. More details are in the original issue #103

[Crazybus]

I’m glad you got it working and thanks for following up!