kucoin watch_order_book initial snapshot error

• OS: ubuntu 16.04
• Programming Language version: python 3.6.5
• CCXT version: 0.1.67

Happens intermittently when trying to subscribe to many pairs. Once it happens the stream never recovers. Would be great if there was a way to detect stale streams and automatically restart

Having trouble reproducing locally, I think it’s related to rate limiting where kucoin will return a blank response or something. Is it possible to add a retry on the initial book snapshot?

enableRateLimiting is set to True here

  File "/python3.6/site-packages/ccxtpro/kucoin.py", line 227, in watch_order_book
    return await self.after(future, self.limit_order_book, symbol, limit, params)
  File "/python3.6/site-packages/ccxtpro/base/exchange.py", line 77, in after
    return method(await future, *args)
  File "/python3.6/site-packages/ccxtpro/base/exchange.py", line 80, in after_async
    return await method(await future, *args)
  File "/python3.6/site-packages/ccxtpro/kucoin.py", line 111, in subscribe
    return await self.watch(url, messageHash, request, messageHash, subscription)
  File "/python3.6/site-packages/ccxtpro/kucoin.py", line 243, in fetch_order_book_snapshot
    previousSequence = sequenceStart - 1
TypeError: unsupported operand type(s) for -: 'NoneType' and 'int'

[kroitor]

Looks like this works, awesome thanks!

Yep, basically, you can play with those settings to adjust for the networking environment. Let me know if you experience any further difficulties with it. In the meantime we will try to address that in a more robust way. Thx for your involvement, appreciate it!

[jaseisen105]

@jaseisen105 we have added one more solution that should also mitigate the problem (it’s already available in 0.1.69, so you have it already), you can add a warmup delay to make sure that the snapshot sequence number is older than the first delta:

exchange = ccxtpro.kucoin({
    'enableRateLimit': True,
    'options': {
        'fetchOrderBookSnapshot': {
            'maxAttempts': 1000,  # max attempts to fetch the snapshot
            'delay': 1000,  # a warmup delay in milliseconds before attempting to fetch the snapshot
        },
    },
})

↑ Adding a warmup delay should also help increase the reliability of synchronization with KuCoin. Let me know if that helps or not.

(I got the email with the logs from you, public feeds do not include sensitive info). The problem is that KuCoin returns a snapshot too old, compared to the WS delta feed, probably due to caching on the webservers – so, an added delay should help). Please, try values from 1000-5000 (1-5 seconds) for the 'delay' and let me know if that helps or not.

Thx!

Looks like this works, awesome thanks!

@jaseisen105 and another quick question – do you have any caching http/https proxies on the production system? (just to rule them out).

None