AWS RedHat - cluster networking issues/lags using canal and flannel plugins

Describe the bug The curl command for k8s deployed services on AWS RHEL takes about 1-2 seconds using calico plugin while for canal and flannel it takes 1-3 minutes.

 [ec2-user@ec2-35-180-45-41 ~]$ time curl -o /dev/null -s -w '%{http_code}' -k http://google.pl
 301
 real    0m0.044s
 user    0m0.002s
 sys     0m0.003s
 
 [ec2-user@ec2-35-180-45-41 ~]$ time curl 'http://127.0.0.1:32300/ignite?cmd=version'
 {"successStatus":0,"error":null,"sessionToken":null,"response":"2.5.0"}
 real    1m3.267s
 user    0m0.001s
 sys     0m0.009s
 
 [ec2-user@ec2-35-180-45-41 ~]$ time curl -o /dev/null -s -w '%{http_code}' -k https://ec2-35-180-45-41:30104/auth/
 200
 real    1m3.468s
 user    0m0.043s
 sys     0m0.077s

To Reproduce Steps to reproduce the behavior:

1. execute epicli init ... (with params)
2. edit config file
3. execute epicli apply ...

Expected behavior A clear and concise description of what you expected to happen.

Config files Configuration that should be included in the yaml file:

• PostgreSQL - at least 1 vm

---
kind: configuration/kubernetes-master
name: default
provider: aws
specification:
  advanced:
    networking:
      plugin: canal

---
kind: configuration/applications
title: "Kubernetes Applications Config"
provider: aws
name: default
specification:
  applications:
  - name: rabbitmq
    enabled: yes
    image_path: rabbitmq:3.7.10
    #image_pull_secret_name: regcred # optional
    service:
      name: rabbitmq-cluster
      port: 30672
      management_port: 31672
      replicas: 2
      namespace: queue
    rabbitmq:
      #amqp_port: 5672 #optional - default 5672
      plugins: # optional list of RabbitMQ plugins
        - rabbitmq_management_agent
        - rabbitmq_management
      policies: # optional list of RabbitMQ policies
        - name: ha-policy2
          pattern: ".*"
          definitions:
            ha-mode: all
      custom_configurations: #optional list of RabbitMQ configurations (new format -> https://www.rabbitmq.com/configure.html)
        - name: vm_memory_high_watermark.relative
          value: 0.5
      #cluster:
        #is_clustered: true #redundant in in-Kubernetes installation, it will always be clustered
        #cookie: "cookieSetFromDataYaml" #optional - default value will be random generated string
  - name: ignite-stateless
    enabled: yes
    image_path: "apacheignite/ignite:2.5.0" # it will be part of the image path: {{local_repository}}/{{image_path}}
    namespace: ignite
    service:
      rest_nodeport: 32300
      sql_nodeport: 32301
      thinclients_nodeport: 32302
    replicas: 2
    enabled_plugins:
    - ignite-kubernetes # required to work on K8s
    - ignite-rest-http
  - name: auth-service # requires PostgreSQL to be installed in cluster
    enabled: yes
    image_path: jboss/keycloak:9.0.0
    use_local_image_registry: true
    #image_pull_secret_name: regcred
    service:
      name: as-testauthdb
      port: 30104
      replicas: 2
      namespace: namespace-for-auth
      admin_user: auth-service-username
      admin_password: PASSWORD_TO_CHANGE
    database:
      name: auth-database-name
      #port: "5432" # leave it when default
      user: auth-db-user
      password: PASSWORD_TO_CHANGE

OS (please complete the following information):

• OS: RHEL7

Cloud Environment (please complete the following information):

• Cloud Provider AWS

Additional context

[mkyc]

Added task for this.

[rafzei]

I would go with disabling only tx-checksum: ethtool -K flannel.1 tx-checksum-ip-generic off. Looks like its sufficient. We can do this ‘live’ and we can easily change it back. The issue is already fixed in https://github.com/kubernetes/kubernetes/pull/92035 - Changelog v1.18.5 We should upgrade k8s to at least 1.18.5 or latest(maybe 1.19 will be there) in Epiphany 0.8