"The Service "kube-dns" is invalid" on fresh install

inspection-report-20190806_185828.tar.gz

During a fresh installation on Bionic with microk8s v1.15.2, I received the following when enabling the dns plugin:

$ microk8s.enable dashboard storage dns
Applying manifest
secret/kubernetes-dashboard-certs created
serviceaccount/kubernetes-dashboard created
role.rbac.authorization.k8s.io/kubernetes-dashboard-minimal created
rolebinding.rbac.authorization.k8s.io/kubernetes-dashboard-minimal created
deployment.apps/kubernetes-dashboard created
service/kubernetes-dashboard created
service/monitoring-grafana created
service/monitoring-influxdb created
service/heapster created
deployment.extensions/monitoring-influxdb-grafana-v4 created
serviceaccount/heapster created
clusterrolebinding.rbac.authorization.k8s.io/heapster created
configmap/heapster-config created
configmap/eventer-config created
deployment.extensions/heapster-v1.5.2 created

If RBAC is not enabled access the dashboard using the default token retrieved with:

token=$(microk8s.kubectl -n kube-system get secret | grep default-token | cut -d " " -f1)
microk8s.kubectl -n kube-system describe secret $token

In an RBAC enabled setup (microk8s.enable RBAC) you need to create a user with restricted
permissions as shown in https://github.com/kubernetes/dashboard/wiki/Creating-sample-user

Enabling default storage class
deployment.extensions/hostpath-provisioner created
storageclass.storage.k8s.io/microk8s-hostpath created
serviceaccount/microk8s-hostpath created
clusterrole.rbac.authorization.k8s.io/microk8s-hostpath created
clusterrolebinding.rbac.authorization.k8s.io/microk8s-hostpath created
Storage will be available soon
Enabling DNS
Applying manifest
serviceaccount/coredns created
configmap/coredns created
deployment.apps/coredns created
clusterrole.rbac.authorization.k8s.io/coredns created
clusterrolebinding.rbac.authorization.k8s.io/coredns created
The Service "kube-dns" is invalid: spec.clusterIP: Invalid value: "10.152.183.10": provided IP is already allocated

That IP address is already assigned to heapster:

$ microk8s.kubectl get services -n kube-system
NAME                   TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)             AGE
heapster               ClusterIP   10.152.183.10    <none>        80/TCP              20m
kubernetes-dashboard   ClusterIP   10.152.183.151   <none>        443/TCP             20m
monitoring-grafana     ClusterIP   10.152.183.72    <none>        80/TCP              20m
monitoring-influxdb    ClusterIP   10.152.183.115   <none>        8083/TCP,8086/TCP   20m

microk8s.status reports that DNS is enabled, but I can’t edit its configmap:

$ microk8s.kubectl -n kube-system edit configmap/kube-dns
Error from server (NotFound): configmaps "kube-dns" not found

Disabling DNS fails:

$ microk8s.disable dns
Disabling DNS
Reconfiguring kubelet
Removing DNS manifest
serviceaccount "coredns" deleted
configmap "coredns" deleted
deployment.apps "coredns" deleted
clusterrole.rbac.authorization.k8s.io "coredns" deleted
clusterrolebinding.rbac.authorization.k8s.io "coredns" deleted
Error from server (NotFound): error when deleting "/home/ubuntu/snap/microk8s/743/tmp/temp.coredns.yaml": services "kube-dns" not found

Attempting to re-enable DNS results in the same error:

$ microk8s.enable dns
Enabling DNS
Applying manifest
serviceaccount/coredns created
configmap/coredns created
deployment.apps/coredns created
clusterrole.rbac.authorization.k8s.io/coredns created
clusterrolebinding.rbac.authorization.k8s.io/coredns created
The Service "kube-dns" is invalid: spec.clusterIP: Invalid value: "10.152.183.10": provided IP is already allocated

[balchua]

@ktsakalozos oh i get it now. Thanks for the insight. What about enabling dns by default upon installing microk8s. I think most apps needs the dns anyway.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.