calico-node-xxxx “Hit error connecting to datastore” in a newly setup worker node

I want to create a multi-node cluster with two machines on the same private network. So I followed the documentation. Exeuting kubectl get nodes shows that the newly created worker node is indeed Ready.

However I can see the following error in an automatically created pod by the new worker

Hit error connecting to datastore - retry error=Get “https://10.152.183.1:443/api/v1/nodes/foo”: dial tcp 10.152.183.1:44 ││ 3: i/o timeout`

I describe this is more details here.

Output of microk8s inspect inspection-report-20220427_122038.tar.gz

[usersina]

I finally managed to get it working! Running microk8s kubectl get nodes -o wide shows that the Internal IP being used is always the WAN ip and not the LAN one. Therefore I tried looking for a way to change the IP addresses of the nodes and workers and stumbled upon this issue.

At the end, I had to do this for all nodes

microk8s stop
# or for workers: sudo snap stop microk8s

sudo vim.tiny /var/snap/microk8s/current/args/kubelet
# Add this to bottom: --node-ip=<this-specific-node-lan-ip>

sudo vim.tiny /var/snap/microk8s/current/args/kube-apiserver
# Add this to bottom: --advertise-address=<this-specific-node-lan-ip>

microk8s start
# or for workers: sudo snap start microk8s

That said, being able to specify both --node-ip and --advertise-address through runtime envs is definitely helpful.

[usersina]

Closing the issue since this here is a direct fix for it.