Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
Trial name could not be passed to env
See original GitHub issue/kind bug
What steps did you take and what happened: I am trying to get access to the trial name from the trial’s pod (I need it to build S3 bucket path for trained model). I follow this example. But I’ve got an error
Error from server: error when creating "trial-metadata-substitution.yaml": admission webhook "validator.experiment.katib.kubeflow.org" denied the request: parameter reference ${trialSpec.Name} does not exist in spec.parameters: [{lr double {0.03 0.01 [] }}]
This example itself does not work.
What did you expect to happen: Trials running with env TRIAL_NAME resolved as trail’s name.
Anything else you would like to add: This example was working on Katib 0.12.0. It looks like additional check added in this PR might cause this issue. One of the resolution might be to skip this check here if it’s trial metadata (such as name, kind, apiversion etc.).
But I will be happy with any other solution which allows me to pass trial name to pod as environment variable.
Environment:
- Katib version (check the Katib controller image version): 0.13.0
- Kubernetes version: 1.22
- OS : CentOS
Impacted by this bug? Give it a 👍 We prioritize the issues with the most 👍
Issue Analytics
- State:
- Created a year ago
- Reactions:3
- Comments:6 (5 by maintainers)
Top Results From Across the Web
Top Related Medium Post
Top Related StackOverflow Question
Troubleshoot Live Code
Top Related Reddit Thread
Top Related Hackernoon Post
Top Related Tweet
Top Related Dev.to Post
Top Related Hashnode Post
Instead of skipping metadata, we can validate the metadata fields.
We need to move the some of the validations from https://github.com/kubeflow/katib/blob/master/pkg/controller.v1beta1/experiment/manifest/generator.go#L123 to verify fields
I’m sorry that I didn’t consider the parameter substitution case.
As @alexeygorobets said, I think we can skip parameter reference checking when it’s a trial metadata. Please let me know if there’s anything I can help with.