Microk8s network implementation is faulted. SNAT issue.

Default installation of Microk8s fails test from “Kubernetes tutorial” described here: https://kubernetes.io/docs/tutorials/services/source-ip/

Full output of test below:

px2@sm3:~$ kubectl config set-context --current --namespace=test
Context "microk8s" modified.
px2@sm3:~$ kubectl create deployment source-ip-app --image=k8s.gcr.io/echoserver:1.4

deployment.apps/source-ip-app created
px2@sm3:~$ 
px2@sm3:~$ kubectl expose deployment source-ip-app --name=clusterip --port=80 --target-port=8080
service/clusterip exposed
px2@sm3:~$ kubectl run busybox -it --image=busybox --restart=Never --rm^C
px2@sm3:~$ kubectl get svc
NAME        TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
clusterip   ClusterIP   10.152.183.84   <none>        80/TCP    17s
px2@sm3:~$ kubectl run busybox -it --image=busybox --restart=Never --rm
If you don't see a command prompt, try pressing enter.
/ # ip a 
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
3: eth0@if7: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1450 qdisc noqueue 
    link/ether 06:e8:ba:a1:40:d9 brd ff:ff:ff:ff:ff:ff
    inet 10.1.95.3/24 scope global eth0
       valid_lft forever preferred_lft forever
/ # wget -qO - 10.152.183.84
CLIENT VALUES:
client_address=10.1.95.1
command=GET
real path=/
query=nil
request_version=1.1
request_uri=http://10.152.183.84:8080/

SERVER VALUES:
server_version=nginx: 1.10.0 - lua: 10001

HEADERS RECEIVED:
connection=close
host=10.152.183.84
user-agent=Wget
BODY:
-no body in request-/ # 

IP address expected to be presented is busybox pod IP address - 10.1.95.3 IP address presented as source reach out service - 10.1.95.1

This means that packets are being SNATed. A side effect of this bug is inability of using revDNS inside cluster.

Inspection report attached. inspection-report-20200424_164356.tar.gz

[peix2]

inspection-report-20200428_192812.tar.gz

[peix2]

Tested on 2 node deployment where server and client pods are on different nodes and all works as it should. Problem solved. Again, thanks @ktsakalozos for taking care of it.