Namerd not updated automatically

Hello,

We are using namerd for dynamic dtab, but when we want to update a route, change are not take into account without restart of namerd pod.

Kubernetes version:

NAME                                           STATUS    AGE       VERSION   EXTERNAL-IP   OS-IMAGE                      KERNEL-VERSION
ip-xxx.eu-central-1.compute.internal   Ready     7d        v1.7.6    <none>        Debian GNU/Linux 8 (jessie)   4.4.78-k8s
ip-xxx.eu-central-1.compute.internal   Ready     15d       v1.7.6    <none>        Debian GNU/Linux 8 (jessie)   4.4.78-k8s

Deployed over AWS.

Linkerd Configuration:

    # Custom HTTP Ingress Controller listening on port 80
    # Accessible Externaly
    - protocol: http
      label: http-ingress
      servers:
        - port: 80
          ip: 0.0.0.0
          clearContext: true
      interpreter:
        kind: io.l5d.namerd
        dst: /#/io.l5d.k8s/linkerd/4100/namerd
        namespace: external
      identifier:
        kind: io.l5d.path
        segments: 1
        consume: true

Namerd Configuration:

    admin:
      ip: 0.0.0.0
      port: 9991

    namers:
    - kind: io.l5d.k8s
      experimental: true
      host: localhost
      port: 8001

    storage:
      kind: io.l5d.k8s
      host: localhost
      port: 8001
      namespace: linkerd

    interfaces:
    - kind: io.l5d.thriftNameInterpreter
      ip: 0.0.0.0
      port: 4100
    - kind: io.l5d.httpController
      ip: 0.0.0.0
      port: 4180

Namerd Service:

apiVersion: v1
kind: Service
metadata:
  name: namerd
  namespace: linkerd
spec:
  selector:
    app: namerd
  type: LoadBalancer
  ports:
  - name: thrift
    port: 4100
  - name: http
    port: 4180
  - name: admin
    port: 9991

Linkerd Metrics:

metrics.txt

[activeshadow]

@klingerf so far the fix looks to be working! I can restart pods, causing them to get a new endpoint IP, and linkerd picks it up and continues to send them data. w00t!

Thanks!

[wmorgan]

Update: we have identified a possible root cause and are working to verify. This is a high priority issue.