ActorFuture roadmap

Hi!

Currently ActorFutures are not really Futures, but just a caching mechanism. As explained here: https://distributed.dask.org/en/latest/actors.html#call-remote-methods,

They can’t be placed into additional submit or map calls to form dependencies.

For my use case, I want to have a dag, in which every part of the dag is persistent. So I need to pass the output of 1 part of the dag (=ActorFuture) as a parameter to the submit call of the next part. Unfortunately this is not possible yet. TypeError: ('Could not serialize object of type ActorFuture.', '<ActorFuture>') (I know in similar projects like Ray, this is possible). I was wondering if someone knows what the plans with Dask Actor(Future)s are? Will ActorFuture be fully featured Futures soon or…?

Thanks for helping! Pj

[mrocklin]

Yes, this work seems in scope. It’s challenging because the scheduler doesn’t keep track of ActorFutures. ActorFutures are intentionally not tracked by the scheduler because there can be many more of them (billions), so there would need to be some way to promote an ActorFuture into a normal future that didn’t affect the common case.

On Mon, Aug 12, 2019 at 9:29 AM Tom Augspurger [e-mail] wrote:

@PidgeyBE https://github.com/PidgeyBE are you interested in working on this? Either way, can you provide a short code snippet demonstrating the issue?

Matt can correct me if I’m wrong, but it seems like it would be in scope since it fits with our handling of Future objects.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/dask/distributed/issues/2950?email_source=notifications&email_token=AACKZTF7OJPVXR3NOLOQTC3QEFQUZA5CNFSM4IK643B2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4CQ5PA#issuecomment-520425148, or mute the thread https://github.com/notifications/unsubscribe-auth/AACKZTAHU37FCFAWQSX3QK3QEFQUZANCNFSM4IK643BQ .

[TomAugspurger]

Not that I know of. Are you interested in working on it @jherreral?