Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Interaction between ray actors and multiprocessing

See original GitHub issue

I would want to combine ray together with python multiprocessing and was wondering, is it possible to have ray actors own a multiprocessing.Queue? I’d want to initialize the actors with such a queue and wonder if one can expect it to work. Obviously, if the actor is on another node it wouldn’t work, but if the worker process is on the same node as the main process.

Issue Analytics

State:
Created 5 years ago
Comments:14 (8 by maintainers)

Top GitHub Comments

3reactions

bioniclescommented, Jul 14, 2020

It would be hugely useful to have a simple solution for this sort of producer-consumer pattern (env-agent) where producers and consumers are agent pools

2reactions

frthjfcommented, Oct 2, 2020

Hi @crypdick, unfortunately not, but it’s been a while since I’ve last worked on this. I believe ray.util.queue was not available back then, so might be worth a new try?

Top Results From Across the Web

Distributed multiprocessing.Pool — Ray 2.2.0

Ray supports running distributed python programs with the multiprocessing.Pool API using Ray Actors instead of local processes. This makes it easy to scale ......

Modern Parallel and Distributed Python: A Quick Tutorial on Ray

Ray takes the existing concepts of functions and classes and translates them to the distributed setting as tasks and actors . This API...

Modern Parallel and Distributed Python: A Quick Tutorial on Ray

Ray takes the existing concepts of functions and classes and translates them to the distributed setting as tasks and actors.

Writing your First Distributed Python Application with Ray

Ray provides actors to allow you to parallelize an instance of a class. Code wise, all you need to add to a Python...

4. Remote Actors - Scaling Python with Ray [Book] - O'Reilly

Much like Rayâ€™s remote functions, all Ray actors are remote actors, even when running on the same machine. In a nutshell, an actor...