Async Read Stream Data from buffer

Hello,

I went through the documentation and examples how to read data from the stream, and I would like to ask a question.

I found out there is the function pop_stream_data_from_stream_buffer and I saw data can be read from the stream with this snippet:

    msg = bwsm.pop_stream_data_from_stream_buffer(book_ticker_id)
    if msg:
        book_ticker(msg)
    msg = bwsm.pop_stream_data_from_stream_buffer(user_stream_id)
    if msg:
        order_status(msg)
    time.sleep(0.01)

Would it be possible to read the data in an asynchronous way without doing the “time.sleep” ?

Thank you 😃

[jon4hz]

Hoi @brianmori

I was working on a script to use UBWA in combination with asyncio but truth be told, I’m not an expert on asyncio so I am really not sure if it’s best practice. I was playing around with the following code:

from __future__ import print_function
from unicorn_binance_websocket_api.unicorn_binance_websocket_api_manager import BinanceWebSocketApiManager
import threading, os, time, asyncio


binance_websocket_api_manager = BinanceWebSocketApiManager(exchange="binance.com")


async def print_stream_data_from_stream_buffer(binance_websocket_api_manager):
    print("waiting 5 seconds, then we start flushing the stream_buffer")
    await asyncio.sleep(5)
    while True:
        if binance_websocket_api_manager.is_manager_stopping():
            exit(0)
        oldest_stream_data_from_stream_buffer = binance_websocket_api_manager.pop_stream_data_from_stream_buffer()
        if oldest_stream_data_from_stream_buffer is False:
            await asyncio.sleep(0.01)
        else:
            try:
                print(oldest_stream_data_from_stream_buffer)

            except Exception as e:
                # Any kind of error...
                # not able to process the data? write it back to the stream_buffer
                print(f"Error: {e}")
                binance_websocket_api_manager.add_to_stream_buffer(oldest_stream_data_from_stream_buffer)


def run_socket_function(loop, binance_websocket_api_manager):
    # set the event loop and execute it
    asyncio.set_event_loop(loop)
    loop.run_until_complete(print_stream_data_from_stream_buffer(binance_websocket_api_manager))


if __name__ == "__main__":
    # create event loop
    loop = asyncio.new_event_loop()
    
    # subscripe to the socket
    binance_websocket_api_manager.create_stream('miniTicker', 'BTCUSDT', output='dict')

    # start a worker process to process to move the received stream_data from the stream_buffer to a print function
    worker_thread = threading.Thread(target=run_socket_function, args=(loop, binance_websocket_api_manager))
    worker_thread.start()
    time.sleep(5)

[jon4hz]

That’s the same reason I wrote my async code, haha. But I don’t think that you can use an async function as a callback. However you can use a “normal” function as callback to add the ingestion function as a coroutine to the event loop.

from unicorn_binance_websocket_api.unicorn_binance_websocket_api_manager import BinanceWebSocketApiManager
import asyncio


async def ingestion(stream_data, stream_buffer_name=False):
    # filter of events
    print(stream_data)


def callback(stream_data, stream_buffer_name=False): 
    asyncio.ensure_future(ingestion(stream_data, stream_buffer_name))


async def main():
    bsm = BinanceWebSocketApiManager(exchange="binance.com", process_stream_data=callback)
    channels = ['trade', 'kline_1m']
    markets = ['bnbusdt', 'wtcusdt']
    bsm.create_stream(channels, markets, output='dict')
    


if __name__ == "__main__":
    loop = asyncio.new_event_loop()
    loop.run_until_complete(main())