question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Helm chart fails completely

See original GitHub issue

1. Describe Pods

$  kubectl describe pod superset-6d55cbd6cd-n4g26 -n superset


Name:         superset-6d55cbd6cd-n4g26
Namespace:    superset
Priority:     0
Node:         ip-XXXXXXXXXXXXXXXXXX.internal/XXXXXXXXXXXXXXX
Start Time:   Tue, 20 Oct 2020 19:00:33 -0400
Labels:       app=superset
              io.cattle.field/appId=superset
              pod-template-hash=6d55cbd6cd
              release=superset
Annotations:  kubernetes.io/psp: eks.privileged
Status:       Running
IP:           172.77.103.174
IPs:
  IP:           172.77.103.174
Controlled By:  ReplicaSet/superset-6d55cbd6cd
Init Containers:
  wait-for-postgres:
    Container ID:  docker://6f69681cfae08d9e718387584743c9b5016f685709d308026d91c80ddcfae1df
    Image:         busybox:latest
    Image ID:      docker-pullable://busybox@sha256:a9286defaba7b3a519d585ba0e37d0b2cbee74ebfe590960b0b1d6a5e97d1e1d
    Port:          <none>
    Host Port:     <none>
    Command:
      /bin/sh
      -c
      until nc -zv $DB_HOST $DB_PORT -w1; do echo 'waiting for db'; sleep 1; done
    State:          Terminated
      Reason:       Completed
      Exit Code:    0
      Started:      Tue, 20 Oct 2020 19:00:34 -0400
      Finished:     Tue, 20 Oct 2020 19:00:34 -0400
    Ready:          True
    Restart Count:  0
    Environment Variables from:
      superset-env  Secret  Optional: false
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-47hg6 (ro)
Containers:
  superset:
    Container ID:  docker://2be8afd1300a905c7bc5fcb4638d2b9a8bf224ecccd2020203fa992514e42150
    Image:         preset/superset:latest
    Image ID:      docker-pullable://preset/superset@sha256:211d58cfbefa1daa0cb0ca2f850bef62b99ab730575ee9b6bdfa394a2cb9d031
    Port:          8088/TCP
    Host Port:     0/TCP
    Command:
      /bin/sh
      -c
      . /app/pythonpath/superset_bootstrap.sh; /usr/bin/docker-entrypoint.sh
    State:          Waiting
      Reason:       CrashLoopBackOff
    Last State:     Terminated
      Reason:       Error
      Exit Code:    3
      Started:      Wed, 21 Oct 2020 10:29:17 -0400
      Finished:     Wed, 21 Oct 2020 10:29:20 -0400
    Ready:          False
    Restart Count:  185
    Environment Variables from:
      superset-env  Secret  Optional: false
    Environment:
      SUPERSET_PORT:  8088
    Mounts:
      /app/pythonpath from superset-config (ro)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-47hg6 (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  superset-config:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  superset-config
    Optional:    false
  default-token-47hg6:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-47hg6
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason   Age                   From                                     Message
  ----     ------   ----                  ----                                     -------
  Warning  BackOff  80s (x4249 over 15h)  kubelet, ip-XXXXXXXXXXXXXX.internal  Back-off restarting failed container

2. Kubectl Logs:

$ kubectl logs -f superset-6d55cbd6cd-n4g26 -n superset

: not found /app/pythonpath/superset_bootstrap.sh:
: not found /app/pythonpath/superset_bootstrap.sh:
ERROR: Invalid requirement: ''
WARNING: You are using pip version 20.2.3; however, version 20.2.4 is available.
You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.
: not found /app/pythonpath/superset_bootstrap.sh:
[2020-10-21 14:29:18 +0000] [13] [INFO] Starting gunicorn 20.0.4
[2020-10-21 14:29:18 +0000] [13] [INFO] Listening at: http://0.0.0.0:8088 (13)
[2020-10-21 14:29:18 +0000] [13] [INFO] Using worker: gthread
[2020-10-21 14:29:18 +0000] [16] [INFO] Booting worker with pid: 16
Found but failed to import local superset_config
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/cachelib/redis.py", line 41, in __init__
    import redis
ModuleNotFoundError: No module named 'redis'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/app/superset/config.py", line 956, in <module>
    import superset_config  # pylint: disable=import-error
  File "/app/pythonpath/superset_config.py", line 39, in <module>
    key_prefix='superset_results'
  File "/usr/local/lib/python3.7/site-packages/cachelib/redis.py", line 43, in __init__
    raise RuntimeError('no redis module found')
RuntimeError: no redis module found
Failed to create app
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/cachelib/redis.py", line 41, in __init__
    import redis
ModuleNotFoundError: No module named 'redis'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/app/superset/app.py", line 59, in create_app
    app.config.from_object(config_module)
  File "/usr/local/lib/python3.7/site-packages/flask/config.py", line 174, in from_object
    obj = import_string(obj)
  File "/usr/local/lib/python3.7/site-packages/werkzeug/utils.py", line 568, in import_string
    __import__(import_name)
  File "/app/superset/config.py", line 956, in <module>
    import superset_config  # pylint: disable=import-error
  File "/app/pythonpath/superset_config.py", line 39, in <module>
    key_prefix='superset_results'
  File "/usr/local/lib/python3.7/site-packages/cachelib/redis.py", line 43, in __init__
    raise RuntimeError('no redis module found')
RuntimeError: no redis module found
[2020-10-21 14:29:19 +0000] [16] [ERROR] Exception in worker process
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/cachelib/redis.py", line 41, in __init__
    import redis
ModuleNotFoundError: No module named 'redis'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
    worker.init_process()
  File "/usr/local/lib/python3.7/site-packages/gunicorn/workers/gthread.py", line 92, in init_process
    super().init_process()
  File "/usr/local/lib/python3.7/site-packages/gunicorn/workers/base.py", line 119, in init_process
    self.load_wsgi()
  File "/usr/local/lib/python3.7/site-packages/gunicorn/workers/base.py", line 144, in load_wsgi
    self.wsgi = self.app.wsgi()
  File "/usr/local/lib/python3.7/site-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "/usr/local/lib/python3.7/site-packages/gunicorn/app/wsgiapp.py", line 49, in load
    return self.load_wsgiapp()
  File "/usr/local/lib/python3.7/site-packages/gunicorn/app/wsgiapp.py", line 39, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/usr/local/lib/python3.7/site-packages/gunicorn/util.py", line 411, in import_app
    app = app(*args, **kwargs)
  File "/app/superset/app.py", line 69, in create_app
    raise ex
  File "/app/superset/app.py", line 59, in create_app
    app.config.from_object(config_module)
  File "/usr/local/lib/python3.7/site-packages/flask/config.py", line 174, in from_object
    obj = import_string(obj)
  File "/usr/local/lib/python3.7/site-packages/werkzeug/utils.py", line 568, in import_string
    __import__(import_name)
  File "/app/superset/config.py", line 956, in <module>
    import superset_config  # pylint: disable=import-error
  File "/app/pythonpath/superset_config.py", line 39, in <module>
    key_prefix='superset_results'
  File "/usr/local/lib/python3.7/site-packages/cachelib/redis.py", line 43, in __init__
    raise RuntimeError('no redis module found')
RuntimeError: no redis module found
[2020-10-21 14:29:19 +0000] [16] [INFO] Worker exiting (pid: 16)
[2020-10-21 14:29:20 +0000] [13] [INFO] Shutting down: Master
[2020-10-21 14:29:20 +0000] [13] [INFO] Reason: Worker failed to boot

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:15 (6 by maintainers)

github_iconTop GitHub Comments

5reactions
salinaaaaaacommented, Nov 3, 2020

@mistercrunch plz see above — I’ve left multiple messages in slack channel. You didn’t do the right things to try & reproduce the issue. I’m a bit frustrated tbh: You also didn’t answer any of my questions to clarify carries in things.

1reaction
issue-label-bot[bot]commented, Oct 21, 2020

Issue-Label Bot is automatically applying the label #bug to this issue, with a confidence of 0.84. Please mark this comment with 👍 or 👎 to give our bot feedback!

Links: app homepage, dashboard and code for this bot.

Read more comments on GitHub >

github_iconTop Results From Across the Web

How to check reason for Helm install failure - Stack Overflow
I think that the simplest solution is to add the --debug flag for the installation command: helm install chart my-chart --debug.
Read more >
Helm chart fails completely · Issue #11363 · apache/superset
helm install superset . After a few min, the thing was up :) Verified by running: kubectl port-forward svc/superset 8088:8088 ...
Read more >
Troubleshooting - Helm
The old Google helm chart repository has been replaced by a new Helm chart repository. Run the following command to permanently fix this:...
Read more >
Helm charts cannot roll back after upgrade fails - IBM
When a chart failed to upgrade, sometimes after you fix the issue that ... The installer saves the latest Helm history record as...
Read more >
Helm Tricks: Input Validation with 'required' and 'fail'
When writing charts with Helm, the Kubernetes package manager, it's often necessary to incorporate some sort of input validation to ensure ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found