Lightrun
question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[7.10.2] Can't create index pattern

See original GitHub issue

Chart version: 7.10.2

Kubernetes version: 1.19.3

Kubernetes provider: E.g. GKE (Google Kubernetes Engine) AKS

Helm Version: Version:“v3.4.2”

helm get release output

e.g. helm get elasticsearch (replace elasticsearch with the name of your helm release)

Output of helm get all elasticsearch 
NAME: elasticsearch
LAST DEPLOYED: Fri Jan 15 10:42:11 2021
NAMESPACE: elastic
STATUS: deployed
REVISION: 1
USER-SUPPLIED VALUES:
antiAffinity: soft
antiAffinityTopologyKey: kubernetes.io/hostname
clusterHealthCheckParams: wait_for_status=green&timeout=1s
clusterName: elasticsearch
esConfig:
  elasticsearch.yml: |
    logger.org.elasticsearch.discovery: DEBUG
    # xpack.security.enabled: true
    # xpack.security.transport.ssl.enabled: true
    # xpack.security.transport.ssl.verification_mode: certificate
    # xpack.security.transport.ssl.keystore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12
    # xpack.security.transport.ssl.truststore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12
    # xpack.security.http.ssl.enabled: true
    # xpack.security.http.ssl.truststore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12
    # xpack.security.http.ssl.keystore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12
esJavaOpts: -Xmx256m -Xms256m
esMajorVersion: ""
extraContainers: ""
extraEnvs:
- name: ELASTIC_PASSWORD
  valueFrom:
    secretKeyRef:
      key: password
      name: elastic-credentials
- name: ELASTIC_USERNAME
  valueFrom:
    secretKeyRef:
      key: username
      name: elastic-credentials
extraInitContainers: ""
extraVolumeMounts: ""
extraVolumes: ""
fsGroup: ""
fullnameOverride: ""
httpPort: 9200
image: docker.elastic.co/elasticsearch/elasticsearch
imagePullPolicy: IfNotPresent
imagePullSecrets: []
imageTag: 7.10.2
ingress:
  annotations: {}
  enabled: false
  hosts:
  - chart-example.local
  path: /
  tls: []
initResources: {}
keystore: []
labels: {}
lifecycle: {}
masterService: ""
masterTerminationFix: false
maxUnavailable: 1
minimumMasterNodes: 1
nameOverride: ""
networkHost: 0.0.0.0
nodeAffinity: {}
nodeGroup: master
nodeSelector: {}
persistence:
  annotations: {}
  enabled: true
podAnnotations: {}
podManagementPolicy: Parallel
podSecurityContext:
  fsGroup: 1000
  runAsUser: 1000
podSecurityPolicy:
  create: false
  name: ""
  spec:
    fsGroup:
      rule: RunAsAny
    privileged: true
    runAsUser:
      rule: RunAsAny
    seLinux:
      rule: RunAsAny
    supplementalGroups:
      rule: RunAsAny
    volumes:
    - secret
    - configMap
    - persistentVolumeClaim
priorityClassName: ""
protocol: http
rbac:
  create: false
  serviceAccountName: ""
readinessProbe:
  failureThreshold: 3
  initialDelaySeconds: 60
  periodSeconds: 30
  successThreshold: 3
  timeoutSeconds: 15
replicas: 2
resources:
  limits:
    cpu: 500m
    memory: 800Mi
  requests:
    cpu: 500m
    memory: 800Mi
roles:
  data: "true"
  ingest: "true"
  master: "true"
schedulerName: ""
secretMounts:
- name: elastic-certificates
  path: /usr/share/elasticsearch/config/certs
  secretName: elastic-certificates
securityContext:
  capabilities:
    drop:
    - ALL
  runAsNonRoot: true
  runAsUser: 1000
service:
  annotations: {}
  httpPortName: http
  labels: {}
  labelsHeadless: {}
  loadBalancerSourceRanges: []
  nodePort: ""
  transportPortName: transport
  type: ClusterIP
sidecarResources: {}
sysctlInitContainer:
  enabled: true
sysctlVmMaxMapCount: 262144
terminationGracePeriod: 120
tolerations: []
transportPort: 9300
updateStrategy: RollingUpdate
volumeClaimTemplate:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi
  storageClassName: managed-premium

COMPUTED VALUES:
antiAffinity: soft
antiAffinityTopologyKey: kubernetes.io/hostname
clusterHealthCheckParams: wait_for_status=green&timeout=1s
clusterName: elasticsearch
enableServiceLinks: true
envFrom: []
esConfig:
  elasticsearch.yml: |
    logger.org.elasticsearch.discovery: DEBUG
    # xpack.security.enabled: true
    # xpack.security.transport.ssl.enabled: true
    # xpack.security.transport.ssl.verification_mode: certificate
    # xpack.security.transport.ssl.keystore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12
    # xpack.security.transport.ssl.truststore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12
    # xpack.security.http.ssl.enabled: true
    # xpack.security.http.ssl.truststore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12
    # xpack.security.http.ssl.keystore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12
esJavaOpts: -Xmx256m -Xms256m
esMajorVersion: ""
extraContainers: ""
extraEnvs:
- name: ELASTIC_PASSWORD
  valueFrom:
    secretKeyRef:
      key: password
      name: elastic-credentials
- name: ELASTIC_USERNAME
  valueFrom:
    secretKeyRef:
      key: username
      name: elastic-credentials
extraInitContainers: ""
extraVolumeMounts: ""
extraVolumes: ""
fsGroup: ""
fullnameOverride: ""
hostAliases: []
httpPort: 9200
image: docker.elastic.co/elasticsearch/elasticsearch
imagePullPolicy: IfNotPresent
imagePullSecrets: []
imageTag: 7.10.2
ingress:
  annotations: {}
  enabled: false
  hosts:
  - chart-example.local
  path: /
  tls: []
initResources: {}
keystore: []
labels: {}
lifecycle: {}
masterService: ""
masterTerminationFix: false
maxUnavailable: 1
minimumMasterNodes: 1
nameOverride: ""
networkHost: 0.0.0.0
nodeAffinity: {}
nodeGroup: master
nodeSelector: {}
persistence:
  annotations: {}
  enabled: true
  labels:
    enabled: false
podAnnotations: {}
podManagementPolicy: Parallel
podSecurityContext:
  fsGroup: 1000
  runAsUser: 1000
podSecurityPolicy:
  create: false
  name: ""
  spec:
    fsGroup:
      rule: RunAsAny
    privileged: true
    runAsUser:
      rule: RunAsAny
    seLinux:
      rule: RunAsAny
    supplementalGroups:
      rule: RunAsAny
    volumes:
    - secret
    - configMap
    - persistentVolumeClaim
priorityClassName: ""
protocol: http
rbac:
  create: false
  serviceAccountAnnotations: {}
  serviceAccountName: ""
readinessProbe:
  failureThreshold: 3
  initialDelaySeconds: 60
  periodSeconds: 30
  successThreshold: 3
  timeoutSeconds: 15
replicas: 2
resources:
  limits:
    cpu: 500m
    memory: 800Mi
  requests:
    cpu: 500m
    memory: 800Mi
roles:
  data: "true"
  ingest: "true"
  master: "true"
  remote_cluster_client: "true"
schedulerName: ""
secretMounts:
- name: elastic-certificates
  path: /usr/share/elasticsearch/config/certs
  secretName: elastic-certificates
securityContext:
  capabilities:
    drop:
    - ALL
  runAsNonRoot: true
  runAsUser: 1000
service:
  annotations: {}
  externalTrafficPolicy: ""
  httpPortName: http
  labels: {}
  labelsHeadless: {}
  loadBalancerIP: ""
  loadBalancerSourceRanges: []
  nodePort: ""
  transportPortName: transport
  type: ClusterIP
sidecarResources: {}
sysctlInitContainer:
  enabled: true
sysctlVmMaxMapCount: 262144
terminationGracePeriod: 120
tolerations: []
transportPort: 9300
updateStrategy: RollingUpdate
volumeClaimTemplate:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi
  storageClassName: managed-premium

HOOKS:
---
# Source: elasticsearch/templates/test/test-elasticsearch-health.yaml
apiVersion: v1
kind: Pod
metadata:
  name: "elasticsearch-azvbj-test"
  annotations:
    "helm.sh/hook": test
    "helm.sh/hook-delete-policy": hook-succeeded
spec:
  securityContext:
    fsGroup: 1000
    runAsUser: 1000
  containers:
  - name: "elasticsearch-vinsr-test"
    image: "docker.elastic.co/elasticsearch/elasticsearch:7.10.2"
    imagePullPolicy: "IfNotPresent"
    command:
      - "sh"
      - "-c"
      - |
        #!/usr/bin/env bash -e
        curl -XGET --fail 'elasticsearch-master:9200/_cluster/health?wait_for_status=green&timeout=1s'
  restartPolicy: Never
MANIFEST:
---
# Source: elasticsearch/templates/poddisruptionbudget.yaml
apiVersion: policy/v1beta1
kind: PodDisruptionBudget
metadata:
  name: "elasticsearch-master-pdb"
spec:
  maxUnavailable: 1
  selector:
    matchLabels:
      app: "elasticsearch-master"
---
# Source: elasticsearch/templates/configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: elasticsearch-master-config
  labels:
    heritage: "Helm"
    release: "elasticsearch"
    chart: "elasticsearch"
    app: "elasticsearch-master"
data:
  elasticsearch.yml: |
    logger.org.elasticsearch.discovery: DEBUG
    # xpack.security.enabled: true
    # xpack.security.transport.ssl.enabled: true
    # xpack.security.transport.ssl.verification_mode: certificate
    # xpack.security.transport.ssl.keystore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12
    # xpack.security.transport.ssl.truststore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12
    # xpack.security.http.ssl.enabled: true
    # xpack.security.http.ssl.truststore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12
    # xpack.security.http.ssl.keystore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12
---
# Source: elasticsearch/templates/service.yaml
kind: Service
apiVersion: v1
metadata:
  name: elasticsearch-master
  labels:
    heritage: "Helm"
    release: "elasticsearch"
    chart: "elasticsearch"
    app: "elasticsearch-master"
  annotations:
    {}
spec:
  type: ClusterIP
  selector:
    release: "elasticsearch"
    chart: "elasticsearch"
    app: "elasticsearch-master"
  ports:
  - name: http
    protocol: TCP
    port: 9200
  - name: transport
    protocol: TCP
    port: 9300
---
# Source: elasticsearch/templates/service.yaml
kind: Service
apiVersion: v1
metadata:
  name: elasticsearch-master-headless
  labels:
    heritage: "Helm"
    release: "elasticsearch"
    chart: "elasticsearch"
    app: "elasticsearch-master"
  annotations:
    service.alpha.kubernetes.io/tolerate-unready-endpoints: "true"
spec:
  clusterIP: None # This is needed for statefulset hostnames like elasticsearch-0 to resolve
  # Create endpoints also if the related pod isn't ready
  publishNotReadyAddresses: true
  selector:
    app: "elasticsearch-master"
  ports:
  - name: http
    port: 9200
  - name: transport
    port: 9300
---
# Source: elasticsearch/templates/statefulset.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: elasticsearch-master
  labels:
    heritage: "Helm"
    release: "elasticsearch"
    chart: "elasticsearch"
    app: "elasticsearch-master"
  annotations:
    esMajorVersion: "7"
spec:
  serviceName: elasticsearch-master-headless
  selector:
    matchLabels:
      app: "elasticsearch-master"
  replicas: 2
  podManagementPolicy: Parallel
  updateStrategy:
    type: RollingUpdate
  volumeClaimTemplates:
  - metadata:
      name: elasticsearch-master
    spec:
      accessModes:
      - ReadWriteOnce
      resources:
        requests:
          storage: 10Gi
      storageClassName: managed-premium
  template:
    metadata:
      name: "elasticsearch-master"
      labels:
        release: "elasticsearch"
        chart: "elasticsearch"
        app: "elasticsearch-master"
      annotations:
        
        configchecksum: bb7cc7c26950ce11b1bd1944ed558b73bb37b4952c3f58a2ef1c6e545ead6a5
    spec:
      securityContext:
        fsGroup: 1000
        runAsUser: 1000
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 1
            podAffinityTerm:
              topologyKey: kubernetes.io/hostname
              labelSelector:
                matchExpressions:
                - key: app
                  operator: In
                  values:
                  - "elasticsearch-master"
      terminationGracePeriodSeconds: 120
      volumes:
        - name: elastic-certificates
          secret:
            secretName: elastic-certificates
        - name: esconfig
          configMap:
            name: elasticsearch-master-config
      enableServiceLinks: true
      initContainers:
      - name: configure-sysctl
        securityContext:
          runAsUser: 0
          privileged: true
        image: "docker.elastic.co/elasticsearch/elasticsearch:7.10.2"
        imagePullPolicy: "IfNotPresent"
        command: ["sysctl", "-w", "vm.max_map_count=262144"]
        resources:
          {}

      containers:
      - name: "elasticsearch"
        securityContext:
          capabilities:
            drop:
            - ALL
          runAsNonRoot: true
          runAsUser: 1000
        image: "docker.elastic.co/elasticsearch/elasticsearch:7.10.2"
        imagePullPolicy: "IfNotPresent"
        readinessProbe:
          exec:
            command:
              - sh
              - -c
              - |
                #!/usr/bin/env bash -e
                # If the node is starting up wait for the cluster to be ready (request params: "wait_for_status=green&timeout=1s" )
                # Once it has started only check that the node itself is responding
                START_FILE=/tmp/.es_start_file

                # Disable nss cache to avoid filling dentry cache when calling curl
                # This is required with Elasticsearch Docker using nss < 3.52
                export NSS_SDB_USE_CACHE=no

                http () {
                  local path="${1}"
                  local args="${2}"
                  set -- -XGET -s

                  if [ "$args" != "" ]; then
                    set -- "$@" $args
                  fi

                  if [ -n "${ELASTIC_USERNAME}" ] && [ -n "${ELASTIC_PASSWORD}" ]; then
                    set -- "$@" -u "${ELASTIC_USERNAME}:${ELASTIC_PASSWORD}"
                  fi

                  curl --output /dev/null -k "$@" "http://127.0.0.1:9200${path}"
                }

                if [ -f "${START_FILE}" ]; then
                  echo 'Elasticsearch is already running, lets check the node is healthy'
                  HTTP_CODE=$(http "/" "-w %{http_code}")
                  RC=$?
                  if [[ ${RC} -ne 0 ]]; then
                    echo "curl --output /dev/null -k -XGET -s -w '%{http_code}' \${BASIC_AUTH} http://127.0.0.1:9200/ failed with RC ${RC}"
                    exit ${RC}
                  fi
                  # ready if HTTP code 200, 503 is tolerable if ES version is 6.x
                  if [[ ${HTTP_CODE} == "200" ]]; then
                    exit 0
                  elif [[ ${HTTP_CODE} == "503" && "7" == "6" ]]; then
                    exit 0
                  else
                    echo "curl --output /dev/null -k -XGET -s -w '%{http_code}' \${BASIC_AUTH} http://127.0.0.1:9200/ failed with HTTP code ${HTTP_CODE}"
                    exit 1
                  fi

                else
                  echo 'Waiting for elasticsearch cluster to become ready (request params: "wait_for_status=green&timeout=1s" )'
                  if http "/_cluster/health?wait_for_status=green&timeout=1s" "--fail" ; then
                    touch ${START_FILE}
                    exit 0
                  else
                    echo 'Cluster is not yet ready (request params: "wait_for_status=green&timeout=1s" )'
                    exit 1
                  fi
                fi
          failureThreshold: 3
          initialDelaySeconds: 60
          periodSeconds: 30
          successThreshold: 3
          timeoutSeconds: 15
        ports:
        - name: http
          containerPort: 9200
        - name: transport
          containerPort: 9300
        resources:
          limits:
            cpu: 500m
            memory: 800Mi
          requests:
            cpu: 500m
            memory: 800Mi
        env:
          - name: node.name
            valueFrom:
              fieldRef:
                fieldPath: metadata.name
          - name: cluster.initial_master_nodes
            value: "elasticsearch-master-0,elasticsearch-master-1,"
          - name: discovery.seed_hosts
            value: "elasticsearch-master-headless"
          - name: cluster.name
            value: "elasticsearch"
          - name: network.host
            value: "0.0.0.0"
          - name: ES_JAVA_OPTS
            value: "-Xmx256m -Xms256m"
          - name: node.data
            value: "true"
          - name: node.ingest
            value: "true"
          - name: node.master
            value: "true"
          - name: node.remote_cluster_client
            value: "true"
          - name: ELASTIC_PASSWORD
            valueFrom:
              secretKeyRef:
                key: password
                name: elastic-credentials
          - name: ELASTIC_USERNAME
            valueFrom:
              secretKeyRef:
                key: username
                name: elastic-credentials
        volumeMounts:
          - name: "elasticsearch-master"
            mountPath: /usr/share/elasticsearch/data

          - name: elastic-certificates
            mountPath: /usr/share/elasticsearch/config/certs
          - name: esconfig
            mountPath: /usr/share/elasticsearch/config/elasticsearch.yml
            subPath: elasticsearch.yml

NOTES:
1. Watch all cluster members come up.
  $ kubectl get pods --namespace=elastic -l app=elasticsearch-master -w
2. Test cluster health using Helm test.
  $ helm test elasticsearch
Output of helm get all kibana 
NAME: kibana
LAST DEPLOYED: Fri Jan 15 10:42:10 2021
NAMESPACE: elastic
STATUS: deployed
REVISION: 1
TEST SUITE: None
USER-SUPPLIED VALUES:
affinity: {}
elasticsearchHosts: http://elasticsearch-master:9200
elasticsearchURL: ""
envFrom: []
extraContainers: ""
extraEnvs:
- name: NODE_OPTIONS
  value: --max-old-space-size=1800
- name: ELASTICSEARCH_USERNAME
  valueFrom:
    secretKeyRef:
      key: username
      name: elastic-credentials
- name: ELASTICSEARCH_PASSWORD
  valueFrom:
    secretKeyRef:
      key: password
      name: elastic-credentials
- name: KIBANA_ENCRYPTION_KEY
  valueFrom:
    secretKeyRef:
      key: encryptionkey
      name: kibana
extraInitContainers: ""
fullnameOverride: ""
healthCheckPath: /app/kibana
hostAliases: []
httpPort: 5601
image: docker.elastic.co/kibana/kibana
imagePullPolicy: IfNotPresent
imagePullSecrets: []
imageTag: 7.10.2
ingress:
  annotations: {}
  enabled: false
  hosts:
  - host: chart-example.local
    paths:
    - path: /
  tls: []
kibanaConfig:
  kibana.yml: |
    kibana:
      autocompleteTimeout: 10000
    # server.ssl:
    #   enabled: true
    #   key: /usr/share/kibana/config/certs/elastic-certificate.pem
    #   certificate: /usr/share/kibana/config/certs/elastic-certificate.pem
    # xpack.security.encryptionKey: ${KIBANA_ENCRYPTION_KEY}
    elasticsearch:
      requestTimeout: 1200000
      shardTimeout: 0
      # ssl:
      #   certificateAuthorities: /usr/share/kibana/config/certs/elastic-certificate.pem
      #   verificationMode: certificate
labels: {}
lifecycle: {}
nameOverride: ""
nodeSelector: {}
podAnnotations: {}
podSecurityContext:
  fsGroup: 1000
priorityClassName: ""
protocol: http
readinessProbe:
  failureThreshold: 3
  initialDelaySeconds: 10
  periodSeconds: 10
  successThreshold: 3
  timeoutSeconds: 5
replicas: 1
resources:
  limits:
    cpu: 250m
    memory: 1000Mi
  requests:
    cpu: 250m
    memory: 700Mi
secretMounts:
- name: elastic-certificate-pem
  path: /usr/share/kibana/config/certs
  secretName: elastic-certificate-pem
securityContext:
  capabilities:
    drop:
    - ALL
  runAsNonRoot: true
  runAsUser: 1000
serverHost: 0.0.0.0
service:
  annotations: {}
  httpPortName: http
  labels: {}
  loadBalancerIP: ""
  loadBalancerSourceRanges: []
  nodePort: ""
  port: 5601
  type: ClusterIP
serviceAccount: ""
tolerations: []
updateStrategy:
  type: Recreate

COMPUTED VALUES:
affinity: {}
elasticsearchHosts: http://elasticsearch-master:9200
elasticsearchURL: ""
envFrom: []
extraContainers: ""
extraEnvs:
- name: NODE_OPTIONS
  value: --max-old-space-size=1800
- name: ELASTICSEARCH_USERNAME
  valueFrom:
    secretKeyRef:
      key: username
      name: elastic-credentials
- name: ELASTICSEARCH_PASSWORD
  valueFrom:
    secretKeyRef:
      key: password
      name: elastic-credentials
- name: KIBANA_ENCRYPTION_KEY
  valueFrom:
    secretKeyRef:
      key: encryptionkey
      name: kibana
extraInitContainers: ""
fullnameOverride: ""
healthCheckPath: /app/kibana
hostAliases: []
httpPort: 5601
image: docker.elastic.co/kibana/kibana
imagePullPolicy: IfNotPresent
imagePullSecrets: []
imageTag: 7.10.2
ingress:
  annotations: {}
  enabled: false
  hosts:
  - host: chart-example.local
    paths:
    - path: /
  path: /
  tls: []
kibanaConfig:
  kibana.yml: |
    kibana:
      autocompleteTimeout: 10000
    # server.ssl:
    #   enabled: true
    #   key: /usr/share/kibana/config/certs/elastic-certificate.pem
    #   certificate: /usr/share/kibana/config/certs/elastic-certificate.pem
    # xpack.security.encryptionKey: ${KIBANA_ENCRYPTION_KEY}
    elasticsearch:
      requestTimeout: 1200000
      shardTimeout: 0
      # ssl:
      #   certificateAuthorities: /usr/share/kibana/config/certs/elastic-certificate.pem
      #   verificationMode: certificate
labels: {}
lifecycle: {}
nameOverride: ""
nodeSelector: {}
podAnnotations: {}
podSecurityContext:
  fsGroup: 1000
priorityClassName: ""
protocol: http
readinessProbe:
  failureThreshold: 3
  initialDelaySeconds: 10
  periodSeconds: 10
  successThreshold: 3
  timeoutSeconds: 5
replicas: 1
resources:
  limits:
    cpu: 250m
    memory: 1000Mi
  requests:
    cpu: 250m
    memory: 700Mi
secretMounts:
- name: elastic-certificate-pem
  path: /usr/share/kibana/config/certs
  secretName: elastic-certificate-pem
securityContext:
  capabilities:
    drop:
    - ALL
  runAsNonRoot: true
  runAsUser: 1000
serverHost: 0.0.0.0
service:
  annotations: {}
  httpPortName: http
  labels: {}
  loadBalancerIP: ""
  loadBalancerSourceRanges: []
  nodePort: ""
  port: 5601
  type: ClusterIP
serviceAccount: ""
tolerations: []
updateStrategy:
  type: Recreate

HOOKS:
MANIFEST:
---
# Source: kibana/templates/configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: kibana-kibana-config
  labels: 
    app: kibana
    release: "kibana"
    heritage: Helm
data:
  kibana.yml: |
    kibana:
      autocompleteTimeout: 10000
    # server.ssl:
    #   enabled: true
    #   key: /usr/share/kibana/config/certs/elastic-certificate.pem
    #   certificate: /usr/share/kibana/config/certs/elastic-certificate.pem
    # xpack.security.encryptionKey: ${KIBANA_ENCRYPTION_KEY}
    elasticsearch:
      requestTimeout: 1200000
      shardTimeout: 0
      # ssl:
      #   certificateAuthorities: /usr/share/kibana/config/certs/elastic-certificate.pem
      #   verificationMode: certificate
---
# Source: kibana/templates/service.yaml
apiVersion: v1
kind: Service
metadata:
  name: kibana-kibana
  labels: 
    app: kibana
    release: "kibana"
    heritage: Helm
spec:
  type: ClusterIP
  ports:
    - port: 5601
      protocol: TCP
      name: http
      targetPort: 5601
  selector:
    app: kibana
    release: "kibana"
---
# Source: kibana/templates/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: kibana-kibana
  labels: 
    app: kibana
    release: "kibana"
    heritage: Helm
spec:
  replicas: 1
  strategy:
    type: Recreate
  selector:
    matchLabels:
      app: kibana
      release: "kibana"
  template:
    metadata:
      labels:
        app: kibana
        release: "kibana"
      annotations:
        
        configchecksum: 00fe223dec8aa4956dccd07e8866221c027f61d5a296a046fea67a3de34fe33
    spec:
      securityContext:
        fsGroup: 1000
      volumes:
        - name: elastic-certificate-pem
          secret:
            secretName: elastic-certificate-pem
        - name: kibanaconfig
          configMap:
            name: kibana-kibana-config
      containers:
      - name: kibana
        securityContext:
          capabilities:
            drop:
            - ALL
          runAsNonRoot: true
          runAsUser: 1000
        image: "docker.elastic.co/kibana/kibana:7.10.2"
        imagePullPolicy: "IfNotPresent"
        env:
          - name: ELASTICSEARCH_HOSTS
            value: "http://elasticsearch-master:9200"
          - name: SERVER_HOST
            value: "0.0.0.0"
          - name: NODE_OPTIONS
            value: --max-old-space-size=1800
          - name: ELASTICSEARCH_USERNAME
            valueFrom:
              secretKeyRef:
                key: username
                name: elastic-credentials
          - name: ELASTICSEARCH_PASSWORD
            valueFrom:
              secretKeyRef:
                key: password
                name: elastic-credentials
          - name: KIBANA_ENCRYPTION_KEY
            valueFrom:
              secretKeyRef:
                key: encryptionkey
                name: kibana
        readinessProbe:
          failureThreshold: 3
          initialDelaySeconds: 10
          periodSeconds: 10
          successThreshold: 3
          timeoutSeconds: 5
          exec:
            command:
              - sh
              - -c
              - |
                #!/usr/bin/env bash -e

                # Disable nss cache to avoid filling dentry cache when calling curl
                # This is required with Kibana Docker using nss < 3.52
                export NSS_SDB_USE_CACHE=no

                http () {
                    local path="${1}"
                    set -- -XGET -s --fail -L

                    if [ -n "${ELASTICSEARCH_USERNAME}" ] && [ -n "${ELASTICSEARCH_PASSWORD}" ]; then
                      set -- "$@" -u "${ELASTICSEARCH_USERNAME}:${ELASTICSEARCH_PASSWORD}"
                    fi

                    STATUS=$(curl --output /dev/null --write-out "%{http_code}" -k "$@" "http://localhost:5601${path}")
                    if [[ "${STATUS}" -eq 200 ]]; then
                      exit 0
                    fi

                    echo "Error: Got HTTP code ${STATUS} but expected a 200"
                    exit 1
                }

                http "/app/kibana"
        ports:
        - containerPort: 5601
        resources:
          limits:
            cpu: 250m
            memory: 1000Mi
          requests:
            cpu: 250m
            memory: 700Mi
        volumeMounts:
          - name: elastic-certificate-pem
            mountPath: /usr/share/kibana/config/certs
          - name: kibanaconfig
            mountPath: /usr/share/kibana/config/kibana.yml
            subPath: kibana.yml
Output of helm get all filebeat 
NAME: filebeat
LAST DEPLOYED: Fri Jan 15 10:42:10 2021
NAMESPACE: elastic
STATUS: deployed
REVISION: 1
TEST SUITE: None
USER-SUPPLIED VALUES:
affinity: {}
clusterRoleRules:
- apiGroups:
  - ""
  resources:
  - namespaces
  - nodes
  - pods
  verbs:
  - get
  - list
  - watch
daemonset:
  affinity: {}
  annotations: {}
  enabled: true
  envFrom: []
  extraEnvs:
  - name: ELASTICSEARCH_USERNAME
    valueFrom:
      secretKeyRef:
        key: username
        name: elastic-credentials
  - name: ELASTICSEARCH_PASSWORD
    valueFrom:
      secretKeyRef:
        key: password
        name: elastic-credentials
  extraVolumeMounts: []
  extraVolumes: []
  filebeatConfig: {}
  hostNetworking: false
  labels: {}
  nodeSelector: {}
  resources:
    limits:
      cpu: 250m
      memory: 200Mi
    requests:
      cpu: 100m
      memory: 100Mi
  secretMounts:
  - name: elastic-certificate-pem
    path: /usr/share/filebeat/config/certs
    secretName: elastic-certificate-pem
  securityContext:
    privileged: false
    runAsUser: 0
  tolerations: []
dnsConfig: {}
envFrom: []
extraContainers: ""
extraEnvs:
- name: ELASTICSEARCH_USERNAME
  valueFrom:
    secretKeyRef:
      key: username
      name: elastic-credentials
- name: ELASTICSEARCH_PASSWORD
  valueFrom:
    secretKeyRef:
      key: password
      name: elastic-credentials
extraInitContainers: []
extraVolumeMounts: []
extraVolumes: []
filebeatConfig:
  filebeat.yml: |
    filebeat.inputs:
    - type: container
      paths:
        - /var/log/containers/*.log
      processors:
      - add_kubernetes_metadata:
          host: ${NODE_NAME}
          matchers:
          - logs_path:
              logs_path: "/var/log/containers/"
      - drop_event:
          when:
            not:
              equals:
                kubernetes.namespace: "elastic"
    output.elasticsearch:
      # username: '${ELASTICSEARCH_USERNAME}'
      # password: '${ELASTICSEARCH_PASSWORD}'
      protocol: http
      hosts: ["elasticsearch-master:9200"]
      # ssl.certificate_authorities:
      #   - /usr/share/filebeat/config/certs/elastic-certificate.pem
fullnameOverride: ""
hostAliases: []
hostPathRoot: /var/lib
image: docker.elastic.co/beats/filebeat
imagePullPolicy: IfNotPresent
imagePullSecrets: []
imageTag: 7.10.2
labels: {}
livenessProbe:
  exec:
    command:
    - sh
    - -c
    - |
      #!/usr/bin/env bash -e
      curl --fail 127.0.0.1:5066
  failureThreshold: 3
  initialDelaySeconds: 10
  periodSeconds: 10
  timeoutSeconds: 5
managedServiceAccount: true
nameOverride: ""
nodeSelector: {}
podAnnotations: {}
podSecurityContext: {}
priorityClassName: ""
readinessProbe:
  exec:
    command:
    - sh
    - -c
    - |
      #!/usr/bin/env bash -e
      filebeat test output
  failureThreshold: 3
  initialDelaySeconds: 10
  periodSeconds: 10
  timeoutSeconds: 5
replicas: 1
resources: {}
secretMounts:
- name: elastic-certificate-pem
  path: /usr/share/filebeat/config/certs
  secretName: elastic-certificate-pem
serviceAccount: ""
serviceAccountAnnotations: {}
terminationGracePeriod: 30
tolerations: []
updateStrategy: RollingUpdate

COMPUTED VALUES:
affinity: {}
clusterRoleRules:
- apiGroups:
  - ""
  resources:
  - namespaces
  - nodes
  - pods
  verbs:
  - get
  - list
  - watch
daemonset:
  affinity: {}
  annotations: {}
  enabled: true
  envFrom: []
  extraEnvs:
  - name: ELASTICSEARCH_USERNAME
    valueFrom:
      secretKeyRef:
        key: username
        name: elastic-credentials
  - name: ELASTICSEARCH_PASSWORD
    valueFrom:
      secretKeyRef:
        key: password
        name: elastic-credentials
  extraVolumeMounts: []
  extraVolumes: []
  filebeatConfig: {}
  hostNetworking: false
  labels: {}
  nodeSelector: {}
  resources:
    limits:
      cpu: 250m
      memory: 200Mi
    requests:
      cpu: 100m
      memory: 100Mi
  secretMounts:
  - name: elastic-certificate-pem
    path: /usr/share/filebeat/config/certs
    secretName: elastic-certificate-pem
  securityContext:
    privileged: false
    runAsUser: 0
  tolerations: []
dnsConfig: {}
envFrom: []
extraContainers: ""
extraEnvs:
- name: ELASTICSEARCH_USERNAME
  valueFrom:
    secretKeyRef:
      key: username
      name: elastic-credentials
- name: ELASTICSEARCH_PASSWORD
  valueFrom:
    secretKeyRef:
      key: password
      name: elastic-credentials
extraInitContainers: []
extraVolumeMounts: []
extraVolumes: []
filebeatConfig:
  filebeat.yml: |
    filebeat.inputs:
    - type: container
      paths:
        - /var/log/containers/*.log
      processors:
      - add_kubernetes_metadata:
          host: ${NODE_NAME}
          matchers:
          - logs_path:
              logs_path: "/var/log/containers/"
      - drop_event:
          when:
            not:
              equals:
                kubernetes.namespace: "elastic"
    output.elasticsearch:
      # username: '${ELASTICSEARCH_USERNAME}'
      # password: '${ELASTICSEARCH_PASSWORD}'
      protocol: http
      hosts: ["elasticsearch-master:9200"]
      # ssl.certificate_authorities:
      #   - /usr/share/filebeat/config/certs/elastic-certificate.pem
fullnameOverride: ""
hostAliases: []
hostNetworking: false
hostPathRoot: /var/lib
image: docker.elastic.co/beats/filebeat
imagePullPolicy: IfNotPresent
imagePullSecrets: []
imageTag: 7.10.2
labels: {}
livenessProbe:
  exec:
    command:
    - sh
    - -c
    - |
      #!/usr/bin/env bash -e
      curl --fail 127.0.0.1:5066
  failureThreshold: 3
  initialDelaySeconds: 10
  periodSeconds: 10
  timeoutSeconds: 5
managedServiceAccount: true
nameOverride: ""
nodeSelector: {}
podAnnotations: {}
podSecurityContext:
  privileged: false
  runAsUser: 0
priorityClassName: ""
readinessProbe:
  exec:
    command:
    - sh
    - -c
    - |
      #!/usr/bin/env bash -e
      filebeat test output
  failureThreshold: 3
  initialDelaySeconds: 10
  periodSeconds: 10
  timeoutSeconds: 5
replicas: 1
resources:
  limits:
    cpu: 1000m
    memory: 200Mi
  requests:
    cpu: 100m
    memory: 100Mi
secretMounts:
- name: elastic-certificate-pem
  path: /usr/share/filebeat/config/certs
  secretName: elastic-certificate-pem
serviceAccount: ""
serviceAccountAnnotations: {}
terminationGracePeriod: 30
tolerations: []
updateStrategy: RollingUpdate

HOOKS:
MANIFEST:
---
# Source: filebeat/templates/serviceaccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: filebeat-filebeat
  annotations:
  labels:
    app: "filebeat-filebeat"
    chart: "filebeat-7.10.2"
    heritage: "Helm"
    release: "filebeat"
---
# Source: filebeat/templates/configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: filebeat-filebeat-config
  labels:
    app: "filebeat-filebeat"
    chart: "filebeat-7.10.2"
    heritage: "Helm"
    release: "filebeat"
data:
  filebeat.yml: |
    filebeat.inputs:
    - type: container
      paths:
        - /var/log/containers/*.log
      processors:
      - add_kubernetes_metadata:
          host: ${NODE_NAME}
          matchers:
          - logs_path:
              logs_path: "/var/log/containers/"
      - drop_event:
          when:
            not:
              equals:
                kubernetes.namespace: "elastic"
    output.elasticsearch:
      # username: '${ELASTICSEARCH_USERNAME}'
      # password: '${ELASTICSEARCH_PASSWORD}'
      protocol: http
      hosts: ["elasticsearch-master:9200"]
      # ssl.certificate_authorities:
      #   - /usr/share/filebeat/config/certs/elastic-certificate.pem
---
# Source: filebeat/templates/clusterrole.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: filebeat-filebeat-cluster-role
  labels:
    app: "filebeat-filebeat"
    chart: "filebeat-7.10.2"
    heritage: "Helm"
    release: "filebeat"
rules:
- apiGroups:
  - ""
  resources:
  - namespaces
  - nodes
  - pods
  verbs:
  - get
  - list
  - watch
---
# Source: filebeat/templates/clusterrolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: filebeat-filebeat-cluster-role-binding
  labels:
    app: "filebeat-filebeat"
    chart: "filebeat-7.10.2"
    heritage: "Helm"
    release: "filebeat"
roleRef:
  kind: ClusterRole
  name: filebeat-filebeat-cluster-role
  apiGroup: rbac.authorization.k8s.io
subjects:
- kind: ServiceAccount
  name: filebeat-filebeat
  namespace: elastic
---
# Source: filebeat/templates/daemonset.yaml
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: filebeat-filebeat
  labels:
    app: "filebeat-filebeat"
    chart: "filebeat-7.10.2"
    heritage: "Helm"
    release: "filebeat"
spec:
  selector:
    matchLabels:
      app: "filebeat-filebeat"
      release: "filebeat"
  updateStrategy:
    type: RollingUpdate
  template:
    metadata:
      annotations:
        
        configChecksum: 063808c5f8066ddb0eaaabf6206998b2b6a33e5e4e25070853ab2e88a67b03b
      name: "filebeat-filebeat"
      labels:
        app: "filebeat-filebeat"
        chart: "filebeat-7.10.2"
        heritage: "Helm"
        release: "filebeat"
    spec:
      serviceAccountName: filebeat-filebeat
      terminationGracePeriodSeconds: 30
      volumes:
      - name: elastic-certificate-pem
        secret:
          secretName: elastic-certificate-pem
      - name: filebeat-config
        configMap:
          defaultMode: 0600
          name: filebeat-filebeat-config
      - name: data
        hostPath:
          path: /var/lib/filebeat-filebeat-elastic-data
          type: DirectoryOrCreate
      - name: varlibdockercontainers
        hostPath:
          path: /var/lib/docker/containers
      - name: varlog
        hostPath:
          path: /var/log
      - name: varrundockersock
        hostPath:
          path: /var/run/docker.sock
      containers:
      - name: "filebeat"
        image: "docker.elastic.co/beats/filebeat:7.10.2"
        imagePullPolicy: "IfNotPresent"
        args:
        - "-e"
        - "-E"
        - "http.enabled=true"
        livenessProbe:
          exec:
            command:
            - sh
            - -c
            - |
              #!/usr/bin/env bash -e
              curl --fail 127.0.0.1:5066
          failureThreshold: 3
          initialDelaySeconds: 10
          periodSeconds: 10
          timeoutSeconds: 5
        readinessProbe:
          exec:
            command:
            - sh
            - -c
            - |
              #!/usr/bin/env bash -e
              filebeat test output
          failureThreshold: 3
          initialDelaySeconds: 10
          periodSeconds: 10
          timeoutSeconds: 5
        resources:
          limits:
            cpu: 1000m
            memory: 200Mi
          requests:
            cpu: 100m
            memory: 100Mi
        env:
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: NODE_NAME
          valueFrom:
            fieldRef:
              fieldPath: spec.nodeName
        - name: ELASTICSEARCH_USERNAME
          valueFrom:
            secretKeyRef:
              key: username
              name: elastic-credentials
        - name: ELASTICSEARCH_PASSWORD
          valueFrom:
            secretKeyRef:
              key: password
              name: elastic-credentials
        securityContext:
          privileged: false
          runAsUser: 0
        volumeMounts:
        - name: elastic-certificate-pem
          mountPath: /usr/share/filebeat/config/certs
        - name: filebeat-config
          mountPath: /usr/share/filebeat/filebeat.yml
          readOnly: true
          subPath: filebeat.yml
        - name: data
          mountPath: /usr/share/filebeat/data
        - name: varlibdockercontainers
          mountPath: /var/lib/docker/containers
          readOnly: true
        - name: varlog
          mountPath: /var/log
          readOnly: true
        # Necessary when using autodiscovery; avoid mounting it otherwise
        # See: https://www.elastic.co/guide/en/beats/filebeat/7.10/configuration-autodiscover.html
        - name: varrundockersock
          mountPath: /var/run/docker.sock
          readOnly: true

NOTES:
1. Watch all containers come up.
  $ kubectl get pods --namespace=elastic -l app=filebeat-filebeat -w

Describe the bug:

When trying to create an index pattern i get this:

image

(Request Timeout) in kibana logs I get this (multiple times):

{"type":"response","@timestamp":"2021-01-15T09:58:05Z","tags":[],"pid":6,"method":"get","statusCode":200,"req":{"url":"/app/kibana","method":"get","headers":{"host":"localhost:5601","user-agent":"curl/7.61.1","accept":"*/*"}

Steps to reproduce: install elasticsearch, filebeat and kibana with the provided values, do a port-forward, then create an index pattern in kibana.

Things I tried

  • with xpack enabled
  • with xpack disabled
  • with one replica of elasticsearch
  • with 2 replicas of elasticsearch
  • increased the elasticsearch.requestTimeout and elasticsearch.shardTimeout in kibana settings
  • increased the memory size for elasticsearch up to 2Gb (and ‘-Xmx1500m -Xms1500m’)

Expected behavior: That the index pattern is created.

Provide logs and/or server output (if relevant):

{"type":"response","@timestamp":"2021-01-15T09:58:05Z","tags":[],"pid":6,"method":"get","statusCode":200,"req":{"url":"/app/kibana","method":"get","headers":{"host":"localhost:5601","user-agent":"curl/7.61.1","accept":"*/*"}

Any additional context: index status as shown from kibana: image

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:5 (2 by maintainers)

github_iconTop GitHub Comments

1reaction
ebuildycommented, Jan 26, 2021

Are you able to use elasticsearch with Kibana ? try with the dev-tool console first.

0reactions
botelastic[bot]commented, Jun 24, 2021

This issue has been automatically closed because it has not had recent activity since being marked as stale.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Not able to create index pattern in kibana (7.10.2)
Hi, I have a standalone setup of ELK 7.10.2 on CentOS Server. I have ingested data into elasticsearch using logstash, but the problem...
Read more >
How to create index pattern in kibana - YouTube
... as part of this lecture we will see, How to create index pattern in kibana. ... Your browser can't play this video....
Read more >
Kibana index pattern not saved - Stack Overflow
When I click "Kibana > Index Patterns" in the panel, it shows "You have data in Elasticsearch. Now, create an index pattern." I...
Read more >
SGS_KIBANA_USER allow to delete index patterns
So, now user has only sg_kublr_kibana_user role. User cannot delete saved objects/index patterns, its OK; User cannot create Index Pattern. Its ...
Read more >
Creating an Index Pattern to Connect to Elasticsearch
In Kibana, in the Management tab, click Index Patterns. The Index Patterns tab is displayed. Click Add New. The Configure an index pattern...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found
Previous page

[logstash] Add podAffinity

Next page

7.10.1 Elasticsearch pods aren't readying up but curl -k http://localhost:9200/_cluster/health says green