feature request: Standard way for microk8s to communicate about a local registry

Many Kubernetes clusters support a local registry, to make pushing images directly to the cluster fast. But every cluster documents it with manual shell scripts:

• https://kind.sigs.k8s.io/docs/user/local-registry/
• https://microk8s.io/docs/registry-built-in
• https://github.com/rancher/k3d/blob/master/docs/registries.md#using-a-local-registry
• https://docs.openshift.com/container-platform/4.3/registry/accessing-the-registry.html#registry-accessing-directly_accessing-the-registry

Because every cluster is slightly different, this makes it hard for tools to automate this work for users.

The big idea

Clusters should use annotations on the kube-system namespace to document how to configure a registry

Detailed design

I propose 3 new “standard” annotations:

x-k8s.io/registry: the host of the registry (hostname and port), as seen from outside of the cluster x-k8s.io/registry-from-cluster: the host of the registry (hostname and port), as seen from inside the cluster x-k8s.io/registry-help: The URL of documentation where users can read more about how to set a registry up.

Tools that would benefit from local registries should read this annotation and display it to the user

What this means for microk8s

All microk8s clusters would have a registry-help annotation pointing users to https://microk8s.io/docs/registry-built-in

When you run microk8s enable registry, microk8s would automatically add the x-k8s.io/registry annotation to nodes in the cluster.

Prior Art

This proposal is based on what we’ve been recommending to people in Tilt, with a lot of success: https://docs.tilt.dev/choosing_clusters.html#custom-clusters

update 5/4 - changed the proposal to use the kube-system namespace instead of node annotations

[ktsakalozos]

Looks good @nicks. You have my +1 on the PR.

[nicks]

The KEP has been marked implementable! I suspect it will be straightforward to implement, and I think I understand the microk8s internals well enough to send a PR if that would help, but it might take me a little while to get to it.