18.04.2 microk8s issue starting

Brand new install of 18.04.2 along with microk8s which installs fine. Trying to run rancher/vm but either everything sits in Evicted or Pending. Network host is attached to is 10.1.1.0/24 which I changed in the config to be 10.10.1.1/24 as I thought maybe that was the issue. I have tried various resolutions including re-install, changing br0 etc etc. I am sure I am doing something stupid. Thanks for any insight.

inspection-report-20190509_162949.tar.gz

Please run microk8s.inspect and attach the generated tarball to this issue.

We appreciate your feedback. Thank you for using microk8s.

[ktsakalozos]

@crsavage1 this is a great question that should reach our official docs eventually.

In MicroK8s container data are kept under /var/snap/microk8s/common/var/lib/containerd and /var/snap/microk8s/common/run/. If you do a df -h you should see that the filesystem holding those paths (probably the root filesystem) is running low on disk space. As I understand you have some other volumes with free disk space you want to use. To do so you have to configure MicroK8s accordingly. Edit /var/snap/microk8s/current/args/containerd and point the --root and --state to the volume you want to use. Here is an example where we use the /mnt

--config ${SNAP_DATA}/args/containerd.toml
--root /mnt/var/lib/containerd
--state /mnt/run/containerd
--address ${SNAP_COMMON}/run/containerd.sock

You must restart MicroK8s for this configuration to take effect: microk8s.stop, microk8s.start.

[ktsakalozos]

Hi @crsavage1

I see your node is under disk pressure (running low on disk), see microk8s.kubectl describe no.

The limits we set are --eviction-hard="memory.available<100Mi,nodefs.available<1Gi,imagefs.available<1Gi" defined in /var/snap/microk8s/current/args/kubelet. You can update the limits and do a microk8s.stop, microk8s.start.