minute aggregate bars missing

I set up a WebSocket connection to grab 1 min aggregates for a number of stocks:

    alpaca_api = alpaca.REST(
        "key",
        "secretkey",
        api_version="v2",
    )
    stream = alpaca.StreamConn(
        "key", "secretkey",
    )

    symbols = {
        symbol: Scalper(
            api=alpaca_api, symbol=symbol, allotted_liquidity=args.allotted_liquidity
        )
        for symbol in args.symbols
    }

    @stream.on(r"^AM")
    async def on_bars(conn, channel, data):
        logger.info(f"{data.symbol} new bar: {data}")
        if data.symbol in symbols:
            symbols[data.symbol].add_new_closing_price(new_closing_price=data.close)

    @stream.on(r"trade_updates")
    async def on_trade_update(conn, channel, data):
        logger.info(f"trade udpate: {data}")
        symbol = data.order["symbol"]
        if symbol in symbols:
            symbols[symbol].on_trade_update(trade_data=data)

    async def check_for_flatten():
        flatten_time = pd.Timestamp.now(tz="America/New_York").replace(
            hour=15, minute=59, second=0
        )
        while True:
            if pd.Timestamp.now(tz="America/New_York") > flatten_time:
                logger.info("time to flatten then exit")
                alpaca_api.cancel_all_orders()
                alpaca_api.close_all_positions()
                sys.exit(0)
            await asyncio.sleep(20)

    channels = ["trade_updates"] + [f"AM.{symbol}" for symbol in args.symbols]
    loop = stream.loop
    loop.run_until_complete(
        asyncio.gather(stream.subscribe(channels), check_for_flatten(),)
    )
    loop.close()

This is similar to the work done here: https://github.com/alpacahq/example-scalping by @umitanuki. However, when I run the script it doesn’t return bars every minute for all the symbols. Some minute aggregates are skipped for some symbols and it changes for different symbols each minute as well.

I tried this with 5 and 15 symbols and still, the same issue persists. Is this on the polygon side or is there a check to make sure the WebSocket returns info for every minute aggregate? Also, if others have had this issue then could it be best to move to REST API with multiple threads running, one for each symbol?

[connor-roche]

I tried this with the Polygon WS and Alpaca WS and still got the same result. While this issue is much more exaggerated in pre-market and after-market (I get a response from the SPY minute interval WS every 5-8 mins) I am still experiencing the issue during market hours. I ran my script with a number of high volume stocks and had this problem for a decent amount of them. Here are some symbols where I ran into the issue: AMZN / DIS / AMD / SHOP / TSLA / GOOG / GOOGL / ZM / FB just to name a few

Keep in mind it doesn’t happen every minute. Some minutes it might return a bar for TSLA and the next minute it might not. For instance, I ran my script for 22 mins and got 17 TSLA bars back spread out through the 22 mins.

If you would like I can send you my script for greater insight.

Thanks for all your help thus far @jaggas & @shlomikushchi!

[drew887]

Hey closing this one out due to inactivity, and I believe this is todo with the api/ws itself and not so much the SDK here