Can this actually be used in Kubernetes?

I am trying to implement cote in kubernetes on IBM bluemix and am having alot of trouble getting it to work.

The bluemix kubernetes implementation has Calico plugin at network layer and Pods can definitely access each other via IP.

I have put a redis service in place and set cote to use that for discovery which works:

2017-09-13T05:23:41.920741203Z core space requester > service.online idea service space responder#7528508b-a087-414e-ade8-2a95825472e1 on 8002

however once it starts actually messaging it seems to drop back to hostnames that do not resolve inside kubernetes, e.g.

2017-09-13T05:23:46.925401243Z Error: getaddrinfo EAI_AGAIN idea:8000
2017-09-13T05:23:46.925441235Z     at Object._errnoException (util.js:1041:11)
2017-09-13T05:23:46.925447197Z     at errnoException (dns.js:58:15)
2017-09-13T05:23:46.925452669Z     at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:95:26)

I’ve trawled through the implementation of node-discover and can see the hostname getting encoded in the message but I’m unclear if that hostname is used to respond.

Is there a way to insist on using IP addresses for communication rather than hostnames or am I looking down the wrong path here?

[drubin]

@LeoVS09 as mentioned https://github.com/dashersw/cote-workshop/pull/5#issuecomment-482807015

Right now the easiest way to get this working in either docker-swarm or k8s is to use the redis service discovery (with DISCOVERY_HOSTNAME env param).

I can update the example workshop to work k8s with redis, but will only get to that next week.

[NickyTope]

Thanks for looking @drubin and @dashersw I’m glad that what I’ve been coming up with is similar to what you have found.

I have currently deployed successfully using a fork of node-discover (and cote to point at it) with the env.DISCOVERY_HOSTNAME || os.hostname() change above and am using pretty much exactly what you put above for the Deployment yaml:

- name: DISCOVERY_HOSTNAME
    valueFrom:
      fieldRef:
        fieldPath: status.podIP

I also found that multicast is on the way for Calico (and many other of the available network plugins for kube), so am looking forward to going back to less configuration for cote in future, however in my current situation (deployment via IBM bluemix) I don’t have enough control over the host environment to change network plugins so would really appreciate you guys considering this change, or perhaps you can see a better way to go back to useHostNames: false in combination with redis discovery and an environment variable so the correct IP can be reported to redis?

Really appreciate you guys looking into this!