Kucoin: wait for welcome message before sending subscription requests

Exchange Kucoin

Subscription type L3orderbook

Describe the bug For some reason if I try to connect to the kucoin L3 orderbook it will often not send any updates at all. However, this bug is very inconsistent, sometimes it will connect and stream updates other times it won’t at all.

To Reproduce Run the following code in a AWS instance close to kucoin, like ap-northeast-1

/**
 * Prototype for maintaining a Level 3 order book for Kucoin according
 * to the instructions defined here:
 * https://docs.kucoin.com/#full-matchengine-data-level-3
 *
 * This technique uses a Map to store orders. It has efficient updates
 * but will be slow for performing tip of book or snapshot operations.
 *
 * # Example
 * ```javascript
 * */
const ccxws = require("ccxws");
const KucoinOrderBook = require("ccxws/src/orderbooks/KucoinOrderBook");
let market = {id: "BTC-USDT", base: "BTC", quote: "USDT"};
markets=[market, {id: "AMPL-USDT", base: "AMPL", quote: "USDT"}, {id: "SXP-USDT", base: "SXP", quote: "USDT"}]
for (let market of markets) {
    let updates = [];
    let ob;
    const client = new ccxws.Kucoin();
    client.subscribeLevel3Updates(market);
    client.on("l3snapshot", async (snapshot) => {
        console.log("length before", updates.length)
        await sleep(5000);
        if (updates.length === 0){
            console.log("Update length 0 wait longer");
            await sleep(5000);
        }
        console.log("updates after", updates.length);
        ob = new KucoinOrderBook(snapshot, updates);
        console.log("running ", market.id);
    });
    client.on("l3update", update => {
        // enqueue updates until snapshot arrives
        if (!ob) {
            updates.push(update);
            return;
        }
        // validate the sequence and exit if we are out of sync
        if (ob.sequenceId + 1 !== update.sequenceId) {
            console.log(`out of sync, expected ${ob.sequenceId + 1}, got ${update.sequenceId}`);
            //process.exit(1);
        }
        // apply update
        ob.update(update);
    });
}
function sleep(ms) {
    return new Promise(resolve => setTimeout(resolve, ms));
}

The exact error I get is as follow: (node:120179) UnhandledPromiseRejectionWarning: Error: Missing update or Must queue updates prior to snapshot, which is the real problem The weirdest thing is, is that it does work at my home computer (Europe), but my latency is also very high here. However, using an AWS instance it will only connect sometimes?!?!? It might be because the snapshot call is too fast after trying to connect but I have no idea tbh. This especially happens when I try to run multiple coins at the same time (using a new ccxws object for each). Especially the SXP/USDT pair seems to fail often. Could we maybe add a function which retries connecting to the socket if no updates start streaming? EDIT: I am actually thinking that it might just be Kucoin servers that are a little broken, perhaps a function to reconnect with the socket would suffice.

Second EDIT: This is the exact output of the error I meant:

length before 0
length before 21
updates after 181
running  SXP-USDT
Update length 0 wait longer
updates after 1416
(node:121384) UnhandledPromiseRejectionWarning: Error: Missing update
    at new KucoinOrderBook (node_modules/ccxws/src/orderbooks/KucoinOrderBook.js:85:15)
    at KucoinClient.<anonymous> (test.js:29:14)
(Use `node --trace-warnings ...` to show where the warning was created)
(node:121384) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:121384) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
updates after 0
(node:121384) UnhandledPromiseRejectionWarning: Error: Must queue updates prior to snapshot
    at new KucoinOrderBook (node_modules/ccxws/src/orderbooks/KucoinOrderBook.js:59:13)
    at KucoinClient.<anonymous> (test.js:29:14)
(node:121384) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 2)

[WoutervDijk]

@bmancini55 I have found it!!! Because I am so close to Kucoin servers, Kucoin servers can’t handle the subscription event and drops it. I think this happens because right after the socket is connected, we immediately ask for a subscription and the socket might not have been fully ready. So I added a small delay to the _onConnected method (line 223 basic-client.js). This is the code that works for me:

  _onConnected() {
    this.emit("connected");
    setTimeout(() => {    for (let [marketSymbol, market] of this._tickerSubs) {
      this._sendSubTicker(marketSymbol, market);
    }
      for (let [marketSymbol, market] of this._candleSubs) {
        this._sendSubCandles(marketSymbol, market);
      }
      for (let [marketSymbol, market] of this._tradeSubs) {
        this._sendSubTrades(marketSymbol, market);
      }
      for (let [marketSymbol, market] of this._level2SnapshotSubs) {
        this._sendSubLevel2Snapshots(marketSymbol, market);
      }
      for (let [marketSymbol, market] of this._level2UpdateSubs) {
        this._sendSubLevel2Updates(marketSymbol, market);
      }
      for (let [marketSymbol, market] of this._level3UpdateSubs) {
        this._sendSubLevel3Updates(marketSymbol, market);
      }
      this._watcher.start();}, 1000);
  }
}

[WoutervDijk]

Awesome find! Sounds like a good fix. Like they always say RTFM 😄