Incorrectly parsed image references if they contain both a tag and digest
See original GitHub issueDescribe the bug
I use image references in my manifests that contain both a tag and a sha256 digest, e.g.: example.com/library/image:v1.2.3@sha256:abcd...
. This causes the admission webhook to fail with:
Error: INSTALLATION FAILED: admission webhook "connaisseur-svc.connaisseur.svc" denied the request: Unexpected Cosign exception for image "example.com/library/image:v1.2.3@sha256:1d076650cbf2f364235e49203a98015d35443d72e74f462085ea3bcedd25cb4c:v2.15.5": Error: parsing reference: could not parse reference: example.com/library/image:v1.2.3@sha256:1d076650cbf2f364235e49203a98015d35443d72e74f462085ea3bcedd25cb4c:v2.15.5
main.go:62: error during command execution: parsing reference: could not parse reference: example.com/library/image:v1.2.3@sha256:1d076650cbf2f364235e49203a98015d35443d72e74f462085ea3bcedd25cb4c:v2.15.5
So it appears that these images are incorrectly parsed. I believe this is caused in https://github.com/sse-secure-systems/connaisseur/blob/v2.6.2/connaisseur/image.py#L30-L65 as adding the following new test data to https://github.com/sse-secure-systems/connaisseur/blob/v2.6.2/tests/test_image.py#L115 causes the tests to fail:
(
"docker.io/library/image:v1.2.3@sha256:f8816ada742348e1adfcec5c2a180b675bf6e4a294e0feb68bd70179451e1242",
"image",
"v1.2.3",
"f8816ada742348e1adfcec5c2a180b675bf6e4a294e0feb68bd70179451e1242",
"library",
"docker.io",
fix.no_exc(),
),
https://github.com/sse-secure-systems/connaisseur/blob/v2.6.2/connaisseur/image.py#L84 has to be adapted to either return the full tag+digest or just the digest as it is passed to cosign verify later on (https://github.com/sse-secure-systems/connaisseur/blob/v2.6.2/connaisseur/validators/cosign/cosign_validator.py#L304).
Cosign itself does support both tags+digest when verifying, but drops the tag if both a tag+digest are specified e.g.
$ COSIGN_EXPERIMENTAL=1 cosign verify ghcr.io/chgl/fhir-server-exporter:v2.0.2@sha256:0d42bd5680994b2f3e376ef1159ec89ede6b1d59892fcd0f8079672304f99f7f
Verification for ghcr.io/chgl/fhir-server-exporter@sha256:0d42bd5680994b2f3e376ef1159ec89ede6b1d59892fcd0f8079672304f99f7f --
...
Expected behavior
It should be possible to verify image references with both a tag and a digest specified. If both are present, from an implementation point of view it’s probably easiest to drop the tag and just use the digest when passing the image reference.
Optional: To reproduce
- Install connaisseur
- try deploying a manifest containing an image reference with both a tag and digest
- fails with the above error message as the image being passed to cosign is incorrectly constructed
Optional: Versions (please complete the following information as relevant):
- OS: <OS and version>
- Kubernetes Cluster: <cluster type and version>
- Notary Server: <notary and version>
- Container registry: <container registry and version>
- Connaisseur: <connaisseur version>
- Other: <other relevant tools and versions>
Optional: Additional context
As a tiny side note, in https://github.com/sse-secure-systems/connaisseur/blob/v2.6.2/connaisseur/image.py#L84 hard-coding the digest to sha256 could possibly be an issue some time in the future as technically various types of digests are possible, see https://github.com/opencontainers/image-spec/blob/main/descriptor.md#digests - although admittedly I’ve never seen anything but sha256 in the wild.
Issue Analytics
- State:
- Created a year ago
- Comments:8 (7 by maintainers)
Please consider the proposed fix at your earliest convenience, @xopham
For clarity, from the OCI spec:
I’m all for immutability, but I’m a human being first and foremost. When I’m operating my enterprise clusters, my observability tools suite only display a tangled mess of digest references following the mutating webhook implementation.
My goal is to know what is deployed in my cluster without wasting time. My human brain knows that tags are mutable. My human brain knows that tags are not semvers. My human brain prefers a shorthand for identifying a manifest. My human brain is why we wish to include tags AND digests in manifest files together.
You can already sort out the validation based on the immutable part, and it’s perfectly clear that this is a bug that requires an immediate fix. I’ll argue later if your mutating webhook also fails to preserve my tag value, but for now: please consider releasing a fix for this bug at your earliest convenience.
It’s for the humans. Thank you!
An interesting situation and surprising that it exists 😅 Without having delved into the details, it seems to me that we have no standard to follow here and one is generally free to do as one likes. That makes me a bit hesitant to be creative, as it is hard to know what people would expect to happen. If we were to support this, there is essentially 2 questions to answer if tag+digest are provided:
I’d presume that the best way forward may be to pass the whole construct to the validator and go for only validating but not modifying the request as there is a digest and the user intended to have that construct validated and the user would have to take care that it can actually be interpreted by the container runtime which I’d rather not want to take care of.