Possible issues with Kubernetes Ingress, default backends, and lexicographic ordering

Setup

I have been experimenting with the Linkerd Ingress controller. I am trying to deploy a sample calculator service. This service is composed of six services:

• the gateway service for people to submit equation
• the tokenizer service to split the equation into operators and digits
• four generic operator services

Based this blog post, I have tried modeling an ingress object for each service much like the world-v1 and world-v2 (world.v2). This gives the illusion that each service is owned by a team and such a team would want to deploy a newer version of their service. In this case, the default and “v2” (.spec.rules.host) version are actually the same thing.

All Ingress objects look like the following:

---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: "addition-operator"
  namespace: "team-addition-operator"
  annotations:
    kubernetes.io/ingress.class: "linkerd"
  labels:
    app: "addition-operator"
    environment: "preprod"
spec:
  backend:
    serviceName: "addition-operator"
    servicePort: "my-http"
  rules:
  - host: "addition-operator"
    http:
      paths:
      - backend:
          serviceName: "addition-operator"
          servicePort: "my-http"

Having one ingress controller (really, a pod in a daemonset) reading many Ingress objects each with their own non-host “default” backend doesn’t make much sense, so I have omitted the .spec.backend in most of the Ingress Objects. However, I did leave the .spec.backend pointed in the gateway’s Ingress object.

Ingress Objects in alphabetical order:

• addition-operator
• division-operator
• gateway (only this has a .spec.backend)
• multiplication-operator
• subtraction-operator
• tokenizer

Actual Behavior

Note: curl’s default proxy port is 1080. I have added this to my Linkerd Daemonset’s Ports and ConfigMap’s servers section.

core@w2 ~ $ curl --proxy localhost subtraction-operator/operate?args=1,6
404 page not found
core@w2 ~ $ curl --proxy localhost addition-operator/operate?args=1,6
{"value":7}core@w2 ~ $

Logs (from the l5d pod I am proxying to):

I 0409 19:13:53.050 UTC THREAD17: k8s no suitable rule found in addition-operator for request subtraction-operator /operate
I 0409 19:13:53.051 UTC THREAD17: k8s no suitable rule found in division-operator for request subtraction-operator /operate
I 0409 19:13:53.051 UTC THREAD17: k8s using default service IngressPath(None,None,default,gateway,my-http) for request subtraction-operator /operate
I 0409 19:14:00.394 UTC THREAD16: k8s found rule matching addition-operator /operate: IngressPath(Some(addition-operator),None,default,addition-operator,my-http)

Expected Behavior

Given that I am using the host subtraction-operator, there is not way I should be routed to the default backend.

Notes

If I remove the gateway’s .spec.backend, the example works as expected.

[jgensler8]

I think some of the earlier examples of Ingress Controllers gave the responsibility of a “default backend” to the Controller, not to the Ingress Objects. link

args:
- /nginx-ingress-controller
- --default-backend-service=$(POD_NAMESPACE)/default-http-backend

Based on these lines, the “default backend” serves 404 pages.

One use case I was thinking of is an organization with multiple applications (say shoes.com and books.org) that are hosted on the same entry Ingress Controller but eventually end up with different “default” page based on the website. The first Ingress Controller wouldn’t have a default backend. Instead, it would have separate backends based on the Host. Each separate backend (shoes.com and books.org) would have its own organizational Ingress Controller with their branded 404 page. Alternatively, they could use Ingress’s .spec.backend. Similar to what @adleong has stated earlier, it is unclear which option (multiple Ingress Controllers w/ default backend set to .spec.ingress vs Single Controller w/ multiple Ingress Objects) would be the best for such an organization.

[esbie]

A catch-all global default service could end up being a pretty common use case. If multiple folks are confused by this, we should re-evaluate how the identifier evaluates.