question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Updated logic for "ambassador.name" in _helpers.tpl breaks the deployment spec selectors on helm upgrade

See original GitHub issue

Describe the bug We are currently running ambassador helm chart v6.9.4 with following details, Chart name : ‘ambassador’ Release name : ‘ambassador-internal’

Now if we try to upgrade to Emissary-ingress chart v7.4.0 or v7.4.1, we are getting below error,

$ helm upgrade -n ingress-controllers --debug ambassador-internal . --version 7.4.0 -f values.yaml
upgrade.go:121: [debug] preparing upgrade for ambassador-internal
upgrade.go:129: [debug] performing update for ambassador-internal
upgrade.go:296: [debug] creating upgraded release for ambassador-internal
client.go:173: [debug] checking 8 resources for changes
client.go:205: [debug] error updating the resource "ambassador-internal":
         cannot patch "ambassador-internal" with kind Deployment: Deployment.apps "ambassador-internal" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{"app.kubernetes.io/instance":"ambassador-internal", "app.kubernetes.io/name":"ambassador-internal"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable
upgrade.go:355: [debug] warning: Upgrade "ambassador-internal" failed: cannot patch "ambassador-internal" with kind Deployment: Deployment.apps "ambassador-internal" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{"app.kubernetes.io/instance":"ambassador-internal", "app.kubernetes.io/name":"ambassador-internal"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable
Error: UPGRADE FAILED: cannot patch "ambassador-internal" with kind Deployment: Deployment.apps "ambassador-internal" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{"app.kubernetes.io/instance":"ambassador-internal", "app.kubernetes.io/name":"ambassador-internal"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable
helm.go:94: [debug] cannot patch "ambassador-internal" with kind Deployment: Deployment.apps "ambassador-internal" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{"app.kubernetes.io/instance":"ambassador-internal", "app.kubernetes.io/name":"ambassador-internal"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable
helm.sh/helm/v3/pkg/kube.(*Client).Update
        /home/circleci/helm.sh/helm/pkg/kube/client.go:218
helm.sh/helm/v3/pkg/action.(*Upgrade).performUpgrade
        /home/circleci/helm.sh/helm/pkg/action/upgrade.go:310
helm.sh/helm/v3/pkg/action.(*Upgrade).Run
        /home/circleci/helm.sh/helm/pkg/action/upgrade.go:130
main.newUpgradeCmd.func2
        /home/circleci/helm.sh/helm/cmd/helm/upgrade.go:154
github.com/spf13/cobra.(*Command).execute
        /go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:842
github.com/spf13/cobra.(*Command).ExecuteC
        /go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:950
github.com/spf13/cobra.(*Command).Execute
        /go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:887
main.main
        /home/circleci/helm.sh/helm/cmd/helm/helm.go:93
runtime.main
        /usr/local/go/src/runtime/proc.go:203
runtime.goexit
        /usr/local/go/src/runtime/asm_amd64.s:1373
UPGRADE FAILED
main.newUpgradeCmd.func2
        /home/circleci/helm.sh/helm/cmd/helm/upgrade.go:156
github.com/spf13/cobra.(*Command).execute
        /go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:842
github.com/spf13/cobra.(*Command).ExecuteC
        /go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:950
github.com/spf13/cobra.(*Command).Execute
        /go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:887
main.main
        /home/circleci/helm.sh/helm/cmd/helm/helm.go:93
runtime.main
        /usr/local/go/src/runtime/proc.go:203
runtime.goexit
        /usr/local/go/src/runtime/asm_amd64.s:1373

To Reproduce Steps to reproduce the behavior:

  1. Install ambassador helm chart v6.9.4 e.g. helm repo add emissary-ingress https://app.getambassador.io helm repo update helm install -n ingress-controllers --debug ambassador-internal emissary-ingress/ambassador --version 6.9.4 -f values.yaml

  2. Upgrade to emissary-ingress v7.4.1 e.g. helm upgrade -n ingress-controllers --debug ambassador-internal . --version 7.4.1 -f values.yaml

You will get this error,

upgrade.go:121: [debug] preparing upgrade for ambassador-internal
upgrade.go:129: [debug] performing update for ambassador-internal
upgrade.go:296: [debug] creating upgraded release for ambassador-internal
client.go:173: [debug] checking 8 resources for changes
client.go:205: [debug] error updating the resource "ambassador-internal":
         cannot patch "ambassador-internal" with kind Deployment: Deployment.apps "ambassador-internal" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{"app.kubernetes.io/instance":"ambassador-internal", "app.kubernetes.io/name":"ambassador-internal"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable
upgrade.go:355: [debug] warning: Upgrade "ambassador-internal" failed: cannot patch "ambassador-internal" with kind Deployment: Deployment.apps "ambassador-internal" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{"app.kubernetes.io/instance":"ambassador-internal", "app.kubernetes.io/name":"ambassador-internal"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable
Error: UPGRADE FAILED: cannot patch "ambassador-internal" with kind Deployment: Deployment.apps "ambassador-internal" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{"app.kubernetes.io/instance":"ambassador-internal", "app.kubernetes.io/name":"ambassador-internal"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable
helm.go:94: [debug] cannot patch "ambassador-internal" with kind Deployment: Deployment.apps "ambassador-internal" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{"app.kubernetes.io/instance":"ambassador-internal", "app.kubernetes.io/name":"ambassador-internal"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable
helm.sh/helm/v3/pkg/kube.(*Client).Update
        /home/circleci/helm.sh/helm/pkg/kube/client.go:218
helm.sh/helm/v3/pkg/action.(*Upgrade).performUpgrade
        /home/circleci/helm.sh/helm/pkg/action/upgrade.go:310
helm.sh/helm/v3/pkg/action.(*Upgrade).Run
        /home/circleci/helm.sh/helm/pkg/action/upgrade.go:130
main.newUpgradeCmd.func2
        /home/circleci/helm.sh/helm/cmd/helm/upgrade.go:154
github.com/spf13/cobra.(*Command).execute
        /go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:842
github.com/spf13/cobra.(*Command).ExecuteC
        /go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:950
github.com/spf13/cobra.(*Command).Execute
        /go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:887
main.main
        /home/circleci/helm.sh/helm/cmd/helm/helm.go:93
runtime.main
        /usr/local/go/src/runtime/proc.go:203
runtime.goexit
        /usr/local/go/src/runtime/asm_amd64.s:1373
UPGRADE FAILED
main.newUpgradeCmd.func2
        /home/circleci/helm.sh/helm/cmd/helm/upgrade.go:156
github.com/spf13/cobra.(*Command).execute
        /go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:842
github.com/spf13/cobra.(*Command).ExecuteC
        /go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:950
github.com/spf13/cobra.(*Command).Execute
        /go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:887
main.main
        /home/circleci/helm.sh/helm/cmd/helm/helm.go:93
runtime.main
        /usr/local/go/src/runtime/proc.go:203
runtime.goexit
        /usr/local/go/src/runtime/asm_amd64.s:1373

Note values.yaml is updated to reflect similar settings or you may leave that entirely while reproducing the issue.

Expected behavior This should have been a clean upgrade (post making the obvious changes around ambassador custom resources in https://www.getambassador.io/docs/emissary/pre-release/topics/install/upgrade/helm/emissary-1.14/emissary-2.2/)

Versions (please complete the following information):

  • Ambassador: Upgrade from helm chart v6.9.4 to v7.4.0/v7.4.1
  • Kubernetes v1.22

Additional context After searching for the diff in two charts, I found out that changes in logic for definition of “ambassador.name” in _helpers.tpl are causing this issue. Basically if any existing ambassador release name contains ‘ambassador’ in it, this will use .Release.name now. While earlier it was using .Chart.name, hence the error mentioned above.

Ref changes for _helpers.tpl - https://github.com/emissary-ingress/emissary/commit/855e05fe1228231d53f552cab754b02cb37a9c90#diff-453b82870ba74a9df3765bca907c193c9c23fd84e9136dd5dae9ba4f7b3a7fa8

Issue Analytics

  • State:closed
  • Created a year ago
  • Comments:5 (1 by maintainers)

github_iconTop GitHub Comments

2reactions
djslycommented, Jun 27, 2022

@AliceProxy , per @AshutoshNirkhe comment it works, we have tested it with a custom chart. We just don’t feel like keeping a custom chart for ever moving forward, since we don’t plan on updating the name of our Ambassador.

The logic doesn’t make sense as of Today, if we would have called our Ambassador release: my-amba-stack, this would have work , currently there is an hardcoded string ambassador that only affects the previous non Emissary deployments IFF it was called ambassador or contains the string ambassador

Please reconsider or provide a more detailed reason why the PR isn’t valid. Right now I feel that datawire are just playing the card “we don’t have the time/energy to review, hence, here’s our Doc we wrote”. We have spend over two weeks testing all the scenarios and this small change solves all of our problem, it is backward compatible and won’t cause other users to have a hard time with their upgrade.

1reaction
AshutoshNirkhecommented, Jun 15, 2022

For testing purpose, I fetched the chart locally and updated _helpers.tpl to use the old definition for “ambassador.name” and it worked. Can somebody explain the rational behind the change (855e05f#diff-453b82870ba74a9df3765bca907c193c9c23fd84e9136dd5dae9ba4f7b3a7fa8) and if we can either revert it or update in a way so that it won’t break existing ambassador deployments ? Maintaining an updated inline chart just for this purpose doesn’t make much sense IMHO.

At the very least, it should be converted to this IMO which will allows overriding it,

{{- define "ambassador.name" -}}
{{- if .Values.nameOverride -}}
{{- .Values.nameOverride | trunc 63 | trimSuffix "-" -}}
{{- else if contains "ambassador" .Release.Name -}}
{{- .Release.Name | trunc 63 | trimSuffix "-" -}}
{{- else -}}
{{- .Chart.Name | trunc 63 | trimSuffix "-" -}}
{{- end -}}
{{- end -}}
Read more comments on GitHub >

github_iconTop Results From Across the Web

charts/_helpers.tpl at master · helm/charts - GitHub
Create a default fully qualified app name. We truncate at 63 chars because some Kubernetes name fields are limited to this (by the...
Read more >
Chart Development Tips and Tricks - Helm
Covers some of the tips and tricks Helm chart developers have learned while building production-quality charts.
Read more >
Helm Chart Labels and Selectors - Stack Overflow
Because of I am new to using Helm, I have some confusion to use selector and labels. So how should use the labels...
Read more >
Helm: reusable chart – named templates, and a generic chart ...
Create a reusable chart for multiple applications with Helm Named Templates using define, include, toYaml, tpl, and range and while loops.
Read more >
Helm Best Practices and Recommendations - KodeKloud
And, finally, you make a huge update to your app, adding a lot of new ... All variable names in the values.yaml file...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found