Node crashes if timeout (rc: -7) is thrown during init()

Describe the bug If timeout is reached (rc: -7) during the first init() call, I receive the error below and app crashes with code: 139.

Assertion failed: loop->watchers[w->fd] == w

I am sure that app doesn’t use Zookeeper instance until there is a ‘connect’ event (which is never received).

To Reproduce

1. Initialize Zookeeper via .init() or .connect() with a timeout set to a tiny value. E.g. 5ms. (This will produce the same effect as on our server, where timeout is set to 10000ms.)
2. See error.

Error Log

2021-01-11 16:58:46,945:14539:ZOO_INFO@log_env@1102: Client environment:zookeeper.version=zookeeper C client 3.5.8
2021-01-11 16:58:46,945:14539:ZOO_INFO@log_env@1106: Client environment:host.name=project
2021-01-11 16:58:46,945:14539:ZOO_INFO@log_env@1113: Client environment:os.name=Linux
2021-01-11 16:58:46,945:14539:ZOO_INFO@log_env@1114: Client environment:os.arch=4.19.121-linuxkit
2021-01-11 16:58:46,946:14539:ZOO_INFO@log_env@1115: Client environment:os.version=#1 SMP Tue Dec 1 17:50:32 UTC 2020
2021-01-11 16:58:46,946:14539:ZOO_INFO@log_env@1123: Client environment:user.name=(null)
2021-01-11 16:58:46,946:14539:ZOO_INFO@log_env@1131: Client environment:user.home=/home/node
2021-01-11 16:58:46,946:14539:ZOO_INFO@log_env@1143: Client environment:user.dir=/srv
2021-01-11 16:58:46,946:14539:ZOO_INFO@zookeeper_init_internal@1177: Initiating client connection, host=zookeeper:2181 sessionTimeout=5 watcher=0x7f988ef61490 sessionId=0 sessionPasswd=<null> context=0x5623b4960d20 flags=0
2021-01-11 16:58:46,947:14539:ZOO_DEBUG@get_next_server_in_reconfig@1330: [OLD] count=0 capacity=0 next=0 hasnext=0
2021-01-11 16:58:46,947:14539:ZOO_DEBUG@get_next_server_in_reconfig@1333: [NEW] count=1 capacity=16 next=0 hasnext=1
2021-01-11 16:58:46,947:14539:ZOO_DEBUG@get_next_server_in_reconfig@1344: Using next from NEW=172.18.0.2:2181
2021-01-11 16:58:46,947:14539:ZOO_DEBUG@zookeeper_connect@2232: [zk] connect()
Interest in (fd=19, read=true, write=true, timeout=1)
2021-01-11 16:58:46,957:14539:ZOO_WARN@zookeeper_interest@2284: Exceeded deadline by 9ms
zk_timer_cb fired
ping timer went off
2021-01-11 16:58:46,957:14539:ZOO_ERROR@handle_socket_error_msg@2386: Socket [172.18.0.2:2181] zk retcode=-7, errno=110(Operation timed out): connection to 172.18.0.2:2181 timed out (exceeded timeout by 7ms)
2021-01-11 16:58:46,958:14539:ZOO_DEBUG@cleanup@1785: Previous connection=[172.18.0.2:2181] delay=0
yield:zookeeper_interest returned error: -7 - operation timeout
zk_timer_cb fired
ping timer went off
2021-01-11 16:58:46,959:14539:ZOO_WARN@zookeeper_interest@2284: Exceeded deadline by 11ms
2021-01-11 16:58:46,960:14539:ZOO_DEBUG@get_next_server_in_reconfig@1330: [OLD] count=0 capacity=0 next=0 hasnext=0
2021-01-11 16:58:46,960:14539:ZOO_DEBUG@get_next_server_in_reconfig@1333: [NEW] count=1 capacity=16 next=1 hasnext=0
2021-01-11 16:58:46,960:14539:ZOO_DEBUG@get_next_server_in_reconfig@1355: Failed to find either new or old
2021-01-11 16:58:46,960:14539:ZOO_DEBUG@zookeeper_connect@2232: [zk] connect()
Assertion failed: loop->watchers[w->fd] == w (../deps/uv/src/unix/core.c: uv__io_stop: 932)

Expected behavior Timeout doesn’t crash node app and closes client connection, so I could recreate client instance and retry to connect.

Desktop (please complete the following information):

• OS: Linux
• Node.js version: v10.23.0

[sirmonin]

However, adding a setTimeout seemed to help a bit. App is no longer crashing and tries to reconnect.

public connect = () => {
        const host = this.config.get('ZOOKEEPER_HOST');
        const port = this.config.get('ZOOKEEPER_PORT');
        this.zkConfig = {
            connect: `${host}:${port}`,
            timeout: 5,
            debug_level: ZooKeeper.ZOO_LOG_LEVEL_DEBUG,
            host_order_deterministic: false,
        };

        this.client = new ZooKeeper(this.zkConfig);

        setTimeout(() => {
            this.client.init(this.zkConfig);
        }, 1000);
    };

[sirmonin]

@DavidVujic I tested under Node v16.14.2. It looks like yield:zookeeper_interest returned error: -7 - operation timeout no longer causes libuv to fail.

Will keep you posted in case issue repeats itself again. For now the issue is solved by updating to Node v16.14.2 or later.

However, timeout issue still persists (eventhough it doesn’t crash node anymore)

2022-04-29 20:18:04,369:218:ZOO_ERROR@handle_socket_error_msg@2489: Socket 172.19.0.4:2181 zk retcode=-7, errno=110(Operation timed out): connection to 172.19.0.4:2181 timed out (exceeded timeout by 10530ms)
yield:zookeeper_interest returned error: -7 - operation timeout

Update: Node.js v14.19.1 doesn’t crash as well.