Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[Kibana] Readiness returning 404 even after healthCheckPath and kibana.yml.

See original GitHub issue

kibana-7.6.2:

v1.14.10-gke.27 E.g. GKE (Google Kubernetes Engine)

v2.16.6:

helm get release output

REVISION: 1
RELEASED: Tue Apr 28 01:43:53 2020
CHART: kibana-7.6.2
USER-SUPPLIED VALUES:
elasticsearchHosts: http://elasticsearch-master.logging.svc.cluster.local:9200
healthCheckPath: /kbn/app/kibana
kibanaConfig:
  kibana.yml: |
    server:
      basePath: "/kbn"
replicas: 1
resources:
  limits:
    cpu: 1000m
    memory: 800Mi
  requests:
    memory: 500Mi

COMPUTED VALUES:
affinity: {}
elasticsearchHosts: http://elasticsearch-master.logging.svc.cluster.local:9200
elasticsearchURL: ""
extraContainers: ""
extraEnvs:
- name: NODE_OPTIONS
  value: --max-old-space-size=1800
extraInitContainers: ""
fullnameOverride: ""
healthCheckPath: /kbn/app/kibana
httpPort: 5601
image: docker.elastic.co/kibana/kibana
imagePullPolicy: IfNotPresent
imagePullSecrets: []
imageTag: 7.6.2
ingress:
  annotations: {}
  enabled: false
  hosts:
  - chart-example.local
  path: /
  tls: []
kibanaConfig:
  kibana.yml: |
    server:
      basePath: "/kbn"
labels: {}
lifecycle: {}
nameOverride: ""
nodeSelector: {}
podAnnotations: {}
podSecurityContext:
  fsGroup: 1000
priorityClassName: ""
protocol: http
readinessProbe:
  failureThreshold: 3
  initialDelaySeconds: 10
  periodSeconds: 10
  successThreshold: 3
  timeoutSeconds: 5
replicas: 1
resources:
  limits:
    cpu: 1000m
    memory: 800Mi
  requests:
    cpu: 1000m
    memory: 500Mi
secretMounts: []
securityContext:
  capabilities:
    drop:
    - ALL
  runAsNonRoot: true
  runAsUser: 1000
serverHost: 0.0.0.0
service:
  annotations: {}
  labels: {}
  loadBalancerSourceRanges: []
  nodePort: ""
  port: 5601
  type: ClusterIP
serviceAccount: ""
tolerations: []
updateStrategy:
  type: Recreate

HOOKS:
MANIFEST:

---
# Source: kibana/templates/configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: kibana-kibana-config
  labels:
    app: kibana
    release: "kibana"
data:
  kibana.yml: |
    server:
      basePath: "/kbn"
---
# Source: kibana/templates/service.yaml
apiVersion: v1
kind: Service
metadata:
  name: kibana-kibana
  labels:
    app: kibana
    release: "kibana"
    heritage: Tiller
spec:
  type: ClusterIP
  ports:
    - port: 5601
      protocol: TCP
      name: http
      targetPort: 5601
  selector:
    app: kibana
    release: "kibana"
---
# Source: kibana/templates/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: kibana-kibana
  labels:
    app: kibana
    release: "kibana"
spec:
  replicas: 1
  strategy:
    type: Recreate
    
  selector:
    matchLabels:
      app: kibana
      release: "kibana"
  template:
    metadata:
      labels:
        app: kibana
        release: "kibana"
      annotations:
        
        configchecksum: 3cb9df9a00643a476f3202deef65bd8fd46ad7f73646da8739d01cc19dd812f
    spec:
      securityContext:
        fsGroup: 1000
        
      volumes:
        - name: kibanaconfig
          configMap:
            name: kibana-kibana-config
      containers:
      - name: kibana
        securityContext:
          capabilities:
            drop:
            - ALL
          runAsNonRoot: true
          runAsUser: 1000
          
        image: "docker.elastic.co/kibana/kibana:7.6.2"
        imagePullPolicy: "IfNotPresent"
        env:
          - name: ELASTICSEARCH_HOSTS
            value: "http://elasticsearch-master.logging.svc.cluster.local:9200"
          - name: SERVER_HOST
            value: "0.0.0.0"
          - name: NODE_OPTIONS
            value: --max-old-space-size=1800
          
        readinessProbe:
          failureThreshold: 3
          initialDelaySeconds: 10
          periodSeconds: 10
          successThreshold: 3
          timeoutSeconds: 5
          
          exec:
            command:
              - sh
              - -c
              - |
                #!/usr/bin/env bash -e
                http () {
                    local path="${1}"
                    set -- -XGET -s --fail

                    if [ -n "${ELASTICSEARCH_USERNAME}" ] && [ -n "${ELASTICSEARCH_PASSWORD}" ]; then
                      set -- "$@" -u "${ELASTICSEARCH_USERNAME}:${ELASTICSEARCH_PASSWORD}"
                    fi

                    STATUS=$(curl --output /dev/null --write-out "%{http_code}" -k "$@" "http://localhost:5601${path}")
                    if [[ "${STATUS}" -eq 200 ]]; then
                      exit 0
                    fi

                    echo "Error: Got HTTP code ${STATUS} but expected a 200"
                    exit 1
                }

                http "/kbn/app/kibana"
        ports:
        - containerPort: 5601
        resources:
          limits:
            cpu: 1000m
            memory: 800Mi
          requests:
            cpu: 1000m
            memory: 500Mi
          
        volumeMounts:
          - name: kibanaconfig
            mountPath: /usr/share/kibana/config/kibana.yml
            subPath: kibana.yml

Describe the bug: readiness returning 404 , configured healthcheckpoint and server.basepath in kibana.yml Expected behavior: should return 200 Any additional context: kibana.yml

elasticsearchHosts: http://elasticsearch-master.logging.svc.cluster.local:9200
replicas: 1
resources:
  requests:
    memory: 500Mi
  limits:
    cpu: 1000m
    memory: 800Mi
healthCheckPath: "/kbn/app/kibana"
kibanaConfig:
  kibana.yml: |
    server:
      basePath: "/kbn"

Was trying expose the app via nginx ingress , so was setting up the basepath