question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

qasync memory leak

See original GitHub issue

I have an example with pyzmq that is leaking around ~60mb/sec of memory. When I run pyzmq with the standard asyncio event loop everything seems to work fine, but using a QEventLoop I see this problem.

import os
import asyncio
import time
import zmq
import zmq.asyncio
import numpy as np
from multiprocessing import Process
from collections import namedtuple
from PyQt5 import QtWidgets
from qasync import QEventLoop


Addr = namedtuple('Addrs', ['name', 'view'])
addr = Addr('graph', 'tcp://127.0.0.1:5557')


def run_worker():
    ctx = zmq.Context()
    socket = ctx.socket(zmq.PUB)
    socket.bind("tcp://127.0.0.1:5557")
    timestamp = 0

    while True:
        try:
            topic = 'view:graph:_auto_Projection.0.Out'
            socket.send_string(topic, zmq.SNDMORE)
            socket.send_pyobj(timestamp, zmq.SNDMORE)
            timestamp += 1
            msg = np.random.randn(1024, 1024)
            socket.send_pyobj(msg)

            time.sleep(0.1)
        except KeyboardInterrupt:
            break

    ctx.destroy()


async def update():
    ctx = zmq.asyncio.Context()
    sock = ctx.socket(zmq.SUB)
    sock.setsockopt_string(zmq.SUBSCRIBE, 'view:graph:_auto_Projection.0.Out')
    sock.connect(addr.view)

    while True:
        topic = await sock.recv_string()
        heartbeat = await sock.recv_pyobj()
        reply = await sock.recv_pyobj()
        print("PID:", os.getpid(), "RECEIVED:", reply)


if __name__ == "__main__":
    worker = Process(target=run_worker)
    worker.start()

    app = QtWidgets.QApplication([])
    loop = QEventLoop(app)
    asyncio.set_event_loop(loop)
    task = asyncio.ensure_future(update())

    try:
        loop.run_forever()
    finally:
        if not task.done():
            task.cancel()
        loop.close()

    # asyncio.run(update())

Issue Analytics

  • State:open
  • Created 3 years ago
  • Reactions:1
  • Comments:5

github_iconTop GitHub Comments

1reaction
syamajalacommented, Aug 20, 2020

The pyzmq developers responded and this bug does appear to be due to qasync/asyncqt https://github.com/zeromq/pyzmq/issues/1406

1reaction
syamajalacommented, Aug 2, 2020

I swapped out zmq for tcp and its not leaking anymore, so i think this proves the leak is coming from pyzmq.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Issue 34745: asyncio ssl memory leak - Python tracker
I've been trying to track down a leak in aiohttp: ... I think it's some low-level native memory leaked by openssl.
Read more >
New features in Salt 3004 Silicon
Pluggable transports, DeltaProxy, Loader refactoring, Vault Enterprise, VMware extensions, Transactional systems, Salt SSH, Memory leaks ...
Read more >
Memory leak with async for - python - Stack Overflow
My problem is that I'm getting a serious memory leak that crashes the whole app after 40+ requests. This is the code sample:...
Read more >
qasync - PyPI
qasync allows coroutines to be used in PyQt/PySide applications by providing an implementation of the PEP 3156 event-loop. qasync is a fork of...
Read more >
Untitled
... see: https://github.com/QubesOS/qubes-python-qasync.git - Initial package. ... watch: some doc clarifications for -n - Fix memory leak - version 4.1.11 ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found