mpi4py.futures and COMM_WORLD.r(): Blocking calls

I am trying to use mpi4py.futures with MpiProcessPool. When I do so, I aim to use the .send() and .recv methods of COMM_WORLD to send data from the master process to the worker processes. However, the workers never receive any message and block execution. Could you let me know whether MpiProcessPool is not meant to be used with send() and recv()?

[dalcinl]

Why would you like to know that? If the rationale is debugging/logging/profiling purposes then I’ll not object. However, if your algorithm/application will depend on that, then perhaps mpi4py.futures is not the right tool for you.

Would it be enough to know the rank post-mortem, after the task completed? If that is the case, then you just need to make your task return (MPI.COMM_WORLD.rank, task_result).

Could you describe what you are trying to achieve exactly?

With MPICommExecutor you may have a better chance. However, you should use a duplicated communicator to not interfere with the master-worker communication, something like this:

comm = MPI.COMM_WORLD
dup_comm = comm.Dup()
with MPICommExecutor(comm) as executor:
    if executor:
        .... # your code here

[blueskypac]

Sorry to ask one more questions here: Is it also not recommended to use send() and recv() with mpi4py.futures.MPICommExecutor? My goal of sending messages back and forth is basically to know on which node a task will be submitted once a slot becomes available after a future has completed. If three futures complete almost at the same time and a new task will then be submitted to the pool, how can I find out to which rank it will go and thus which node will be used in the cluster?