Stream data not udpated after got event "listenKeyExpired"

Version of this library.

unicorn_fy: 0.11.0 unicorn_binance_local_depth_cache: not found unicorn_binance_rest_api: not found unicorn_binance_trailing_stop_loss: not found unicorn_binance_websocket_api: 1.41.0

Solution to Issue cannot be found in the documentation or other Issues and also occurs in the latest version of this library.

• I checked the documentation and other Issues. I am using the latest version of this library.

Hardware?

VPS or other cloud hosting

Operating System?

Linux

Python version?

Python3.8

Installed packages

Package                       Version
----------------------------- -----------
aiohttp                       3.8.1
aiosignal                     1.2.0
aniso8601                     9.0.1
apprise                       0.9.2
argon2-cffi                   21.1.0
async-timeout                 4.0.2
attrs                         21.2.0
autobahn                      21.3.1
Automat                       20.2.0
backcall                      0.2.0
backports.zoneinfo            0.2.1
bidict                        0.21.4
bleach                        4.1.0
cachetools                    4.2.2
certifi                       2021.10.8
cffi                          1.15.0
charset-normalizer            2.0.7
cheroot                       8.5.2
click                         8.0.3
colorama                      0.4.4
constantly                    15.1.0
cryptography                  35.0.0
cycler                        0.10.0
dateparser                    1.1.0
debugpy                       1.5.1
decorator                     5.1.0
defusedxml                    0.7.1
dnspython                     1.16.0
entrypoints                   0.3
eventlet                      0.30.2
Flask                         1.1.2
Flask-Cors                    3.0.10
Flask-RESTful                 0.3.9
Flask-SocketIO                5.0.1
frozenlist                    1.3.0
greenlet                      1.1.2
gunicorn                      20.1.0
hyperlink                     21.0.0
idna                          3.3
importlib-resources           5.4.0
incremental                   21.3.0
influxdb-client               1.25.0
ipykernel                     6.5.1
ipython                       7.30.0
ipython-genutils              0.2.0
ipywidgets                    7.6.5
itsdangerous                  2.0.1
jaraco.functools              3.3.0
jedi                          0.18.1
Jinja2                        3.0.2
jsonschema                    4.2.1
jupyter                       1.0.0
jupyter-client                7.1.0
jupyter-console               6.4.0
jupyter-core                  4.9.1
jupyterlab-pygments           0.1.2
jupyterlab-widgets            1.0.2
kiwisolver                    1.3.2
Markdown                      3.3.4
MarkupSafe                    2.0.1
matplotlib                    3.4.3
matplotlib-inline             0.1.3
mistune                       0.8.4
more-itertools                8.10.0
multidict                     6.0.2
nbclient                      0.5.9
nbconvert                     6.3.0
nbformat                      5.1.3
nest-asyncio                  1.5.1
notebook                      6.4.6
numpy                         1.21.3
oauthlib                      3.1.1
packaging                     21.3
pandas                        1.3.4
pandocfilters                 1.5.0
parso                         0.8.2
pathlib                       1.0.1
pexpect                       4.8.0
pickleshare                   0.7.5
Pillow                        8.4.0
pip                           22.1.1
prometheus-client             0.12.0
prompt-toolkit                3.0.23
psutil                        5.8.0
psycopg2-binary               2.9.1
ptyprocess                    0.7.0
pyasn1                        0.4.8
pyasn1-modules                0.2.8
pycparser                     2.20
Pygments                      2.10.0
pyOpenSSL                     21.0.0
pyparsing                     3.0.2
pyrsistent                    0.18.0
python-binance                1.0.16
python-dateutil               2.8.2
python-engineio               4.3.0
python-socketio               5.2.1
pytz                          2021.3
pytz-deprecation-shim         0.1.0.post0
PyYAML                        6.0
pyzmq                         22.3.0
qtconsole                     5.2.1
QtPy                          1.11.2
redis                         3.5.3
regex                         2021.10.23
requests                      2.26.0
requests-oauthlib             1.3.0
Rx                            3.2.0
schedule                      1.1.0
Send2Trash                    1.8.0
service-identity              21.1.0
setuptools                    56.0.0
six                           1.16.0
SQLAlchemy                    1.4.15
sqlitedict                    1.7.0
terminado                     0.12.1
testpath                      0.5.0
tornado                       6.1
traitlets                     5.1.1
Twisted                       21.7.0
txaio                         21.2.1
typing-extensions             3.10.0.2
tzdata                        2021.5
tzlocal                       4.0.2
ujson                         4.2.0
unicorn-binance-websocket-api 1.41.0
unicorn-fy                    0.11.0
urllib3                       1.26.7
wcwidth                       0.2.5
webencodings                  0.5.1
websocket-client              1.2.1
websockets                    10.3
Werkzeug                      2.0.2
widgetsnbextension            3.5.2
yarl                          1.7.2
zipp                          3.6.0
zope.interface                5.4.0

Logging output

~/g/binance-trade-bot > grep "REMOVE API_KEY" logs/* 
 ~/g/binance-trade-bot > grep "API_KEY" logs/*
 ~/g/binance-trade-bot > grep "API_SECRET" logs/*
 ~/g/binance-trade-bot > grep "LISTEN_KEY" logs/*
 ~/g/binance-trade-bot > grep "listenKeyExpired" logs/*
logs/crypto_trading.log:2022-05-27 21:11:32,282 - crypto_trading_logger - WARNING - WARN: Not handled 'e' event: {'e': 'listenKeyExpired', 'E': 1653657092253}
logs/crypto_trading.log:2022-05-27 21:11:32,282 - crypto_trading_logger - WARNING - WARN: Not handled 'e' event: {'e': 'listenKeyExpired', 'E': 1653657092253}
logs/crypto_trading.log:2022-05-29 00:20:58,989 - crypto_trading_logger - WARNING - WARN: Not handled 'e' event: {'e': 'listenKeyExpired', 'E': 1653754858963}
logs/crypto_trading.log:2022-05-29 00:20:58,990 - crypto_trading_logger - WARNING - WARN: Not handled 'e' event: {'e': 'listenKeyExpired', 'E': 1653754858964}
logs/crypto_trading.log:2022-05-30 00:42:24,114 - crypto_trading_logger - WARNING - Listen key expired: {'e': 'listenKeyExpired', 'E': 1653842544091}
logs/crypto_trading.log.1:2022-05-26 15:35:31,315 - crypto_trading_logger - WARNING - WARN: Not handled 'e' event: {'e': 'listenKeyExpired', 'E': 1653550531286}
logs/crypto_trading.log.1:2022-05-26 15:35:31,316 - crypto_trading_logger - WARNING - WARN: Not handled 'e' event: {'e': 'listenKeyExpired', 'E': 1653550531287}

Processing method?

process_stream_data

Used endpoint?

binance.com-coin_futures

Issue

I subscribe both !userData and !miniTicker stream, !userData stopped to get new update after got listenKeyExpired event. But last time(2022-05-30 00:42:24,114) both userData/miniTicker are stopped to send update.

There is no significant error/warning logs like “time out”, “connection interrupted” etc. No related code change in recent days, except adding some new log specific for listenKeyExpired event. And the scripts never prompt such information in recent months.

I found API “PUT /fapi/v1/listenKey” not exposed, not sure if we can expose this API and then I can invoke it manually to prevent this event?

[hiliev]

Hi, I’m fighting the same issue here with the user data stream stopping after listenKeyExpired is received. In my case, I have two clients connected to the same Binance account and when one of them calls stop_stream() (as part of its cleanup), the listen key gets deleted (there is a call to self.delete_listen_key_by_stream_id(stream_id) inside stop_stream()). If I understand the Binance docs correctly, there could only be a single listen key per account active at any given moment and its shared between all clients:

POST /fapi/v1/listenKey

Start a new user data stream. The stream will close after 60 minutes unless a keepalive is sent. If the account has an active listenKey, that listenKey will be returned and its validity will be extended for 60 minutes. [emphasis mine]

So, when the second client deletes the listen key before exiting, the first client will no longer receive messages on that listen key. Binance will not disconnect the websocket - it will simply stop sending data.

I tried calling replace_stream() when listenKeyExpired is received, but it does not solve the problem. For now, the only local remedy I see is to comment the delete call in the manager and let the key expire on its own if no other client is keeping it alive.

[dima-dmytruk23]

@netsesame what will happen if, for example, an order is filled on the exchange before the creation, or at the time of the stream creation. the message is lost?

I see 2 solutions to the problem:

1. create 2 streams and periodically restart them at different timings
2. update the key constantly.

The first option is more reliable, in my opinion.