question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[BUG] Helm flyteadmin CrashLoopBackOff

See original GitHub issue

Describe the bug

while using Helm chart v0.18.0 (via flux2) flyteadmin pod enters CrashLoopBackOff with the following (relevant) logging output: i’m trying to do a sandbox setup.

│ sync-cluster-resources {"json":{"src":"clusterresource.go:104"},"level":"fatal","msg":"Failed to sync cluster resources [failed to substitute parameterized value for defaultIamRole: unset or unrecognized ValueFrom: [{EnvVar: FilePat │
│ h:}], failed to substitute parameterized value for defaultIamRole: unset or unrecognized ValueFrom: [{EnvVar: FilePath:}], failed to substitute parameterized value for defaultIamRole: unset or unrecognized ValueFrom: [{EnvVar: FileP │
│ ath:}], Failed to decode namespace config template [ab_project_resource_quota.yaml] for namespace [flyteexamples-development] into a kubernetes object with err: yaml: invalid map key: map[interface {}]interface {}{\"projectQuotaCpu\ │
│ ":interface {}(nil)}, Failed to decode namespace config template [ab_project_resource_quota.yaml] for namespace [flyteexamples-staging] into a kubernetes object with err: yaml: invalid map key: map[interface {}]interface {}{\"projec │
│ tQuotaCpu\":interface {}(nil)}, Failed to decode namespace config template [ab_project_resource_quota.yaml] for namespace [flyteexamples-production] into a kubernetes object with err: yaml: invalid map key: map[interface {}]interfac │
│ e {}{\"projectQuotaCpu\":interface {}(nil)}, Failed to decode namespace config template [ab_project_resource_quota.yaml] for namespace [flytetester-development] into a kubernetes object with err: yaml: invalid map key: map[interface │
│  {}]interface {}{\"projectQuotaCpu\":interface {}(nil)}, Failed to decode namespace config template [ab_project_resource_quota.yaml] for namespace [flytetester-staging] into a kubernetes object with err: yaml: invalid map key: map[i │
│ nterface {}]interface {}{\"projectQuotaCpu\":interface {}(nil)}, Failed to decode namespace config template [ab_project_resource_quota.yaml] for namespace [flytetester-production] into a kubernetes object with err: yaml: invalid map │
│  key: map[interface {}]interface {}{\"projectQuotaCpu\":interface {}(nil)}, Failed to decode namespace config template [ab_project_resource_quota.yaml] for namespace [flytesnacks-development] into a kubernetes object with err: yaml: │
│  invalid map key: map[interface {}]interface {}{\"projectQuotaCpu\":interface {}(nil)}, Failed to decode namespace config template [ab_project_resource_quota.yaml] for namespace [flytesnacks-staging] into a kubernetes object with er │
│ r: yaml: invalid map key: map[interface {}]interface {}{\"projectQuotaCpu\":interface {}(nil)}, Failed to decode namespace config template [ab_project_resource_quota.yaml] for namespace [flytesnacks-production] into a kubernetes obj │
│ ect with err: yaml: invalid map key: map[interface {}]interface {}{\"projectQuotaCpu\":interface {}(nil)}]","ts":"2021-10-25T09:16:23Z"}

Please advise on a set of chart values that would prevent this from happening? every other component of the stack is working fine; thanks!

Expected behavior

flyteadmin pod should start

Additional context to reproduce

No response

Screenshots

No response

Are you sure this issue hasn’t been raised already?

  • Yes

Have you read the Code of Conduct?

  • Yes

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:11 (5 by maintainers)

github_iconTop GitHub Comments

2reactions
mariusrugancommented, Oct 26, 2021

@kumare3 thanks for following up, & thanks for your work, we all know the desired state of moving things can be hard to capture in docs,

sure, i’ll try to come up with a PR, i’ll have a working setup these day and reflect on the dependencies & steps taken vs. the docs.

1reaction
evalsocketcommented, Oct 26, 2021

hi, thanks for taking the time to look into and respond, The sandbox rendered chart, has no reference to cluster_resource_manager as in is disabled, https://github.com/flyteorg/flyte/blob/master/deployment/sandbox/flyte_helm_generated.yaml but maybe i’m mistaking the component that is enabled but not configured and generates the error above. In sandbox we enable the cluster_resource_manager https://github.com/flyteorg/flyte/blob/7aa000728ae71a52aee2d448069a3970b35c30c1/charts/flyte/values-sandbox.yaml#L230

Read more comments on GitHub >

github_iconTop Results From Across the Web

[BUG][OpenSearch Dashboard]-CrashLoopBackOff Error #302
CrashLoopBackOff Error I am deploying an OpenSearch cluster on my local machine, after successfully installing the pods with the helm ...
Read more >
Kubernetes CrashLoopBackOff: What it is, and how to fix it?
CrashLoopBackOff is a Kubernetes state representing a restart loop that is happening in a Pod: a container in the Pod is started, but...
Read more >
How to Start Troubleshooting CrashLoopBackoff Errors in ...
One quick way you can begin troubleshooting a CrashLoopBackoff error is to bypass this error in a separate deployment using a blocking ...
Read more >
nginx pod CrashLoopBackOff for Helm-based Operators
Bug 1984781 - nginx pod CrashLoopBackOff for Helm-based Operators ... Description of problem: After creating a Helm-based Operator following ...
Read more >
Understanding Kubernetes CrashLoopBackoff Events
Let's explore what a CrashLoopBackOff error really means and how you can use kubectl commands to drill down and debug. By the end...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found