question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Does sisl fork on import? Warning with mpi4py

See original GitHub issue

So I occasionally write scripts that are simple to parallelize with mpi4py. What I have discovered is that I occasionally get MPI warnings when importing sisl like so:

from mpi4py import MPI
import sisl

Which gives the following MPI warning:

--------------------------------------------------------------------------
A process has executed an operation involving a call to the
"fork()" system call to create a child process.  Open MPI is currently
operating in a condition that could result in memory corruption or
other system errors; your job may hang, crash, or produce silent
data corruption.  The use of fork() (or system() or other calls that
create child processes) is strongly discouraged.

The process that invoked fork was:

  Local host:          [[22936,1],0] (PID 24519)

If you are *absolutely sure* that your application will successfully
and correctly survive a call to fork(), you may disable this warning
by setting the mpi_warn_on_fork MCA parameter to 0.
--------------------------------------------------------------------------

However, the following two versions don’t result in the warning:

import mpi4py
import sisl
import sisl
from mpi4py import MPI

Also importing other libraries doesn’t seem to trigger the warning, only sisl.

I can’t really figure out why it occurs. It doesn’t stop the script from executing, but I still think it would be nice if it could be avoided regardless of import order.

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:8 (8 by maintainers)

github_iconTop GitHub Comments

1reaction
zerothicommented, Oct 6, 2020
0reactions
jonaslbcommented, Oct 8, 2020

I think I will close my issue here, both tqdm and h5py have solutions in their pipelines and it’s not relevant with OpenMPI 4 anyway.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Importing h5py before mpi4py causes OpenMPI warnings #1079
A process has executed an operation involving a call to the "fork()" system call to create a child process. Open MPI is currently...
Read more >
Received warning about fork incompatible with MPI. In what ...
I am using python2.7 multiprocessing together with mpi4py for my calculation and I received warning on fork being used and is not supported...
Read more >
mpi4py.futures — MPI for Python 3.1.4 documentation
This package provides a high-level interface for asynchronously executing callables on a pool of worker processes using MPI for inter-process communication.
Read more >
Tips and FAQs — Chainer 7.8.0 documentation
This is because MultiprocessIterator creates child processes by the fork system call, which has incompatibilities with the design of MPI and InfiniBand.
Read more >
[R-sig-hpc] what causes this fork warning from openmpi
An MPI process has executed an operation involving a call to the "fork()" system call to create a child process. Open MPI is...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found