Using hostPort

I’m a newbie to k8s and this got me up and running super quick, so thanks for that! I wasn’t able to get hostPort working, and came across this comment:

https://github.com/ubuntu/microk8s/blob/11fe17a5c52055eca1959b65d48510eb488ecd3a/microk8s-resources/actions/ingress.yaml#L82

I can’t use hostNetwork or nodePort for my particular use case. Is that comment still correct? Digging around, it seems like it can work in newer versions of Calico, but requires a portMap plugin, but I don’t really know how to go about installing/enabling that.

[ktsakalozos]

That was really helpful thank you for providing us with some context.

I am sure you have already read that the use of hostPort and hostNetwork is not recommended because (among others) they limit the pod management options you have with Kubernetes. For example, it is not clear to me how would you upgrade without downtime.

Here is a suggestion that is more aligned with the Kubernetes way. You put the container in a deployment:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-deployment
  labels:
    app: mine
spec:
  replicas: 3
  selector:
    matchLabels:
      app: mine
  template:
    metadata:
      labels:
        app: mine
    spec:
      containers:
      - name: mycontainer
        image: myimage:latest
        ports:
        - containerPort: 80
          name: reliable
        - containerPort: 6883
          name: unreliable

You can run as many pods as you want given the hosted services are stateless.

In front of the deployment you put a service.

apiVersion: v1
kind: Service
metadata:
  name: service-name
spec:
  type: NodePort
  ports:
  - name: reliable
    port: 80
    protocol: TCP
    nodePort: 30080
  - name: unreliable
    port: 6883
    protocol: UDP
    nodePort: 36883
  selector:
    app: mine

At this point you can mange the deployment without any restrictions. What we do not like are the ports. You have two options here:

1. Adjust the nodePort port range. This is done with --service-node-port-range argument in the kube-apiserver (https://kubernetes.io/docs/reference/command-line-tools-reference/kube-apiserver/). For microk8s you will need to append the argument to /var/snap/microk8s/current/args/kube-apiserver and restart the api server with sudo systemctl restart snap.microk8s.daemon-apiserver.service. the drawback in this approach is that you may have port conflicts with already running services.

2. Use iptables to forward traffic to the right ports. This is what Kubernetes expects from a loadbalancer. I am sure you are familiar with this approach, just for reference you can see (https://www.cyberciti.biz/faq/linux-port-redirection-with-iptables/). In this approach you can even skip the nodePort entirely. You can expose the service to a fixed ClusterIP (something like 10.152.183.X) and then use iptable rules to forward traffic accordingly.

Important note: port 8080 is already in use by the API server.

[jakecobb]

This may have changed since this issue was opened a couple years ago, but this seems to work today. I have microk8s 1.18.2 and it is using --network=cni with flannel.
The issue linked in the snippet @steveh showed in the original report here indicates hostPort support in the major CNI plugins via the portmap capability and this is present in /var/snap/microk8s/1379/args/cni-network/flannel.conflist:

{
    "name": "microk8s-flannel-network",
    "cniVersion": "0.3.1",
    "plugins": [
      {
        "type": "flannel",
        "name": "flannel-plugin",
        "subnetFile": "/var/snap/microk8s/common/run/flannel/subnet.env",
        "dataDir": "/var/snap/microk8s/common/var/lib/cni/flannel",
        "delegate": {
          "hairpinMode": true,
          "isDefaultGateway": true
        }
      },
      {
        "type": "portmap",
        "capabilities": {"portMappings": true},
        "snat": true
      }
    ]
}

I deployed a pod using hostPort and was able to reach the container successfully using it from the host on 127.0.0.1.