Microk8s install can fail with docker hub rate limiting
See original GitHub issueWhen installing Microk8s from a common infrastructure where many users share the same external IP address, it is very easy to trigger the new DockerHub pull rate limit policy. Once that happens, it is no longer possible to install Microk8s.
Example output from microk8s.kubectl describe pod -n kube-system calico-node-mghbl:
Normal Scheduled 14m default-scheduler Successfully assigned kube-system/calico-node-mghbl to cicd3
Normal Pulling 12m (x4 over 14m) kubelet Pulling image "calico/cni:v3.13.2"
Warning Failed 12m (x4 over 14m) kubelet Error: ErrImagePull
Warning Failed 12m (x4 over 14m) kubelet Failed to pull image "calico/cni:v3.13.2": rpc error: code = Unknown desc = failed to pull and unpack image "docker.io/calico/cni:v3.13.2": failed to copy: httpReaderSeeker: failed open: unexpected status code https://registry-1.docker.io/v2/calico/cni/manifests/sha256:bbf7e3ac3f80d0a356a6c27b095bd313d1106f8ed84f85850816ed79295843c1: 429 Too Many Requests - Server message: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit
Normal BackOff 12m (x7 over 14m) kubelet Back-off pulling image "calico/cni:v3.13.2"
Warning Failed 4m33s (x39 over 14m) kubelet Error: ImagePullBackOff
Issue Analytics
- State:
- Created 3 years ago
- Reactions:2
- Comments:16 (3 by maintainers)
Top Results From Across the Web
Microk8s install can fail with docker hub rate limiting #1711
Another approach to work around the dockerhub late limit is to have containerd login with a dockerhub account. That should involve updating /var ......
Read more >DockerHub download rate limits - MicroK8s
1. Login to DockerHub. The DockerHub rate limits are less strict for authenticated users, though they may still cause problems depending on your...
Read more >Solutions when Kubernetes or MicroK8s encounters Docker ...
In fact, there are many different solutions, and you can see them in the DockerHub download rate limits file on the MicroK8s website....
Read more >Dealing with Docker Hub Rate Limiting
The first thing you might want to do is find out what images from the Docker Hub you're using.
Read more >How to Overcome Docker Hub Pull Limits in a Kubernetes ...
The easiest way to overcome the limit seems to be to subscribe to Docker Hub. Then, you need to set credentials either to...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
Another approach to work around the dockerhub late limit is to have containerd login with a dockerhub account. That should involve updating
/var/snap/microk8s/current/args/containerd-template.toml
and restartingmicrok8s.stop; microk8s.start
. Haven’t tried it though.Side loading images is something that would be very beneficial in the case of off-line deployments. Although I suspect you will have a private registry in such setups.
On the long run we should address the the dockerhub rate limit by moving some critical/often-used images into the https://rocks.canonical.com image registry.
Hi, I’ve also encountered and struggled a lot on this issue and managed to find a way to add auth for docker’s repositry. As mentioned @ktsakalozos, you can edit your
/var/snap/microk8s/current/args/containerd-template.toml
to add authentication to your docker repository. The thing is that containerd’s version has changed since microk8s 1.19 (from 1.2.5 to 1.3.7) so its configuration too. You can configure your authentication like so :The “key” you mention for the “configs” part should match your hostname used in the endpoint array (without “https://”).
Then you should restart your microk8s with
microk8s.stop; microk8s.start
.