Running `nerdctl build` with Rancher Desktop fails to pull base images from custom registry with self-signed cert

For bugs, describe what you’re seeing

Using nerdctl build with Rancher Desktop on macOS involves pulling a base image from a custom registry which uses self-signed corporate cert, and the error is x509: certificate signed by unknown authority. I have the root CA certs in KeyChain as well as under /usr/local/share/ca-certificates on my host machine. I understand that Rancher Desktop has recently added support for installing the host CA certs into k3s under the cover. However, when I check the BuildKit instance running in the kube-image namespace in k3s under the cover of Rancher Desktop, it doesn’t seem to have the corporate root CA certs imported from the host machine.

To Reproduce Steps to reproduce the behaviour:

$ nerdctl build -f Dockerfile .

Result

[+] Building 0.4s (3/3) FINISHED                                                                                                                                                            
 => [internal] load build definition from Dockerfile                                                                                                                            0.1s
 => => transferring dockerfile: 38B                                                                                                                                                    0.0s
 => [internal] load .dockerignore                                                                                                                                                      0.0s
 => => transferring context: 2B                                                                                                                                                        0.0s
 => ERROR [internal] load metadata for foobar.com/myimage:tag                                                                                                             0.2s
------
 > [internal] load metadata for foobar.com/myimage:tag
------
error: failed to solve: failed to solve with frontend dockerfile.v0: failed to create LLB definition: failed to do request: Head https://foobar.com/v2/myimage/manifests/tag: x509: certificate signed by unknown authority
FATA[0000] unrecognized image format            

Setup (please complete the following information):

• OS: macOS 11.6.1
• Rancher Desktop version: 0.6.1
• Kubernetes version: 1.21.5

[shubb30]

I found a workaround, by injecting the corp certs into a secret, and then mounting that into the builder DaemonSet.

$ kubectl create secret generic -n kube-image corp-certs \
                             --from-file=root_ca="/Users/me/certificates/Root-CA.pem" \
                             --from-file=subCA="/Users/me/certificates/subCA.pem"
secret/corp-certs created

Then edit the daemonset: kubectl edit -n kube-image daemonset.apps/builder

Add the secret as a volume:

      volumes:
      ...
      - name: corp-certs
        secret:
          defaultMode: 420
          secretName: corp-certs

Add a volumemount:

        volumeMounts:
        ...
        - mountPath: /usr/local/share/ca-certificates/
          name: corp-certs
          readOnly: true

Add a lifecycle postStart to update the CA certs:

        image: docker.io/moby/buildkit:v0.8.3
        imagePullPolicy: IfNotPresent
        lifecycle:
          postStart:
            exec:
              command:
              - update-ca-certificates

And now, nerdctl/kim can build an image using an image in my private image repo.

[shubb30]

You are correct, the workaround only works until Rancher Desktop is restarted.
@jandubois, @dweomer, I hope Rancher has a plan to fix this before January 31, when Docker Desktop can no longer be used. Rancher Desktop is the best replacement I have found, but this issue makes it unusable for developers.