Exposing microK8s to cloud load balancers

MicroK8s is running on 3 EC2 instances in HA mode with Traefik 2 as ingress controller stuck with <pending> for it’s External IP.

I spent all of Friday writing Terraform rules to create an AWS Network Load Balancer and now I’m uncertain if microK8s supports even the minimum --cloud provider: external flag in which to interface with these externally provided resources. I’m curious if others have gotten this to work. To be clear, I’m using Traefik 2 installed with helm3 but you could test this using the native ingress addon too. You can add this flag to /var/snap/microk8s/current/arg under the kubelet, apiserver and controller files then restart microK8s with microk8s stop and microk8s start.

Now I’m wondering first of all if adding an AWS NLB using Terraform or manually as demonstrated here is even supported and, if it is, if there is perhaps a simpler way, in terms of reducing my own maintenance overhead, such as installing Juju and using its aws-integrator charm, which includes support for AWS Network Load Balancers (provisioning too?). This tutorial uses the kubernetes-core bundle in addition to microK8s to enable AWS integration. It’s unclear if it works with exclusively microk8s clusters as in my scenario.

To be clear, --cloud-provider=aws would be the simplest solution but is unsupported by microK8s and will result in your node failing to switch to Ready status.

One question users might ask is, “why don’t you just use EKS?” To which my answer is, I’m exploring a full-stack Canonical product pipeline (with an eye toward purchasing Ubuntu Advantage) using minimum viable resources on Big Cloud.

[Bessonov]

activity

[joshbranham]

I had to do more than that, the provider ID was needed too (which I get from ec2 metadata)

microk8s kubectl patch node \
    "$host" \
    --patch '{"spec":{"providerID":"'"$providerid"'"}}'

echo "> Setting command line args"
echo "--cloud-provider=aws" >> /var/snap/microk8s/current/args/kube-apiserver
echo "--cloud-provider=aws" >> /var/snap/microk8s/current/args/kubelet
echo "--cloud-provider=aws" >> /var/snap/microk8s/current/args/kube-controller-manager
echo "--provider-id=$providerid" >> /var/snap/microk8s/current/args/kubelet