DNS lookup failure causes cluster connection to hang
See original GitHub issueIn current ioredis (v4.14.1), when DNS resolution fails on connection to one host in a Redis cluster, the cluster connection closes and attempts to reconnect.
With the following test script:
const Redis = require('ioredis');
const hosts = [
"a.valid.cluster.host",
"host.that.does.not.exist"
];
const redis = new Redis.Cluster(hosts.map(h => ({ port: 6379, host: h })));
redis.on('connect', () => console.log('Redis connected'));
redis.on('ready', () => console.log('Redis ready'));
redis.on('error', err => console.error('Redis error', err));
redis.on('end', () => console.log('Redis closed'));
With v3.2.2 - v.4.1.0:
$ node index.js
Redis connected
Redis ready
… and the program waiting in a connected state
With ioredis v4.2.0 - v4.3.0:
$ node index.js
$
… the process exits immediately with no output or errors
With ioredis v4.3.1 - v4.14.1:
$ node index.js
… and program neither connects nor exits.
With debugging in that case, we see
$ DEBUG=ioredis.* node index.js
ioredis:cluster status: [empty] -> connecting +0ms
ioredis:cluster resolved hostname working.host.example.com to IP 127.0.0.1 +5ms
ioredis:cluster failed to resolve hostname does.not.exist to IP: getaddrinfo ENOTFOUND does.not.exist +5ms
ioredis:cluster status: connecting -> close +0ms
ioredis:cluster closed because Error: getaddrinfo ENOTFOUND does.not.exist +0ms
ioredis:cluster status: close -> reconnecting +0ms
ioredis:cluster connecting failed: Error: getaddrinfo ENOTFOUND does.not.exist +4ms
ioredis:cluster Cluster is disconnected. Retrying after 102ms +100ms
ioredis:cluster status: reconnecting -> connecting +0ms
ioredis:cluster resolved hostname working.host.example.com to IP 127.0.0.1 +1ms
ioredis:cluster failed to resolve hostname does.not.exist to IP: getaddrinfo ENOTFOUND does.not.exist +1ms
ioredis:cluster status: connecting -> close +0ms
ioredis:cluster closed because Error: getaddrinfo ENOTFOUND does.not.exist +1ms
ioredis:cluster status: close -> reconnecting +0ms
ioredis:cluster Got error Error: getaddrinfo ENOTFOUND does.not.exist when reconnecting. Ignoring... +0ms
ioredis:cluster Cluster is disconnected. Retrying after 104ms +104ms
That points to the commit https://github.com/luin/ioredis/commit/21138af
In the case of bad DNS lookup on one of many cluster hosts, I would expect the 3.2.2 behavior of connecting using the good hosts or to emit an error. The hanging behavior seems undesirable.
For my operational needs, I would rather see a DNS lookup failure behave like a down server – let the cluster connection succeed without that server.
Issue Analytics
- State:
- Created 4 years ago
- Reactions:1
- Comments:6 (1 by maintainers)
Top GitHub Comments
This is a real and ongoing issue, yes? Shouldn’t allow autoclose?
This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 7 days if no further activity occurs, but feel free to re-open a closed issue if needed.