Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Creation of file with flag="a" and driver="mpio" not working

See original GitHub issue

Hello,

I’m running in a issue introduced by PR #1902 (https://github.com/h5py/h5py/pull/1902/files#diff-6515aaa459980029aa5ae782298c82cacb68f8f6f3218a9f0419f7148bd80f30L208). When opening a non existing file with File(filename, "a", driver="mpio", comm=comm) OSError gets raised. This is because mpi4py manages the opening and it raises OSError instead of FileNotFoundError.

Here an error log

  /opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/site-packages/h5py/_hl/files.py:211: in make_fid
      with File(filename, "a", driver="mpio", comm=comm) as h5f:
  /opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/site-packages/h5py/_hl/files.py:444: in __init__
      fid = make_fid(name, mode, userblock_size,
  /opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/site-packages/h5py/_hl/files.py:211: in make_fid
      fid = h5f.open(name, h5f.ACC_RDWR, fapl=fapl)
  h5py/_objects.pyx:54: in h5py._objects.with_phil.wrapper
      fid = h5f.open(name, h5f.ACC_RDWR, fapl=fapl)
  h5py/_objects.pyx:54: in h5py._objects.with_phil.wrapper
      ???
      ???
  h5py/_objects.pyx:55: in h5py._objects.with_phil.wrapper
  h5py/_objects.pyx:55: in h5py._objects.with_phil.wrapper
      ???
  _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
  
      ???
  _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
  
  >   ???
  E   OSError: Unable to open file (MPI_ERR_NO_SUCH_FILE: no such file or directory)
  
  h5py/h5f.pyx:100: OSError
  >   ???
  E   OSError: Unable to open file (MPI_ERR_NO_SUCH_FILE: no such file or directory)
  
  h5py/h5f.pyx:100: OSError

Issue Analytics

State:
Created 2 years ago
Comments:10 (10 by maintainers)

Top GitHub Comments

1reaction

takluyvercommented, Jul 7, 2021

And the MPI tests:

https://github.com/h5py/h5py/blob/cc34c585ab29e859abda1d99830e7b9868197adb/h5py/tests/test_file.py#L755-L783

1reaction

takluyvercommented, Jul 7, 2021

I think append mode is only tested with the default driver (or we’d have caught this already), but here are some relevant tests you can start with:

https://github.com/h5py/h5py/blob/cc34c585ab29e859abda1d99830e7b9868197adb/h5py/tests/test_file.py#L83-L99

https://github.com/h5py/h5py/blob/cc34c585ab29e859abda1d99830e7b9868197adb/h5py/tests/test_file.py#L228-L316

Top Results From Across the Web

std::fstream doesn't create file - c++ - Stack Overflow

Simply opening it with the flags "binary | in | out | trunc" will work fine. – TorbenC. Feb 6, 2015 at 11:05....

Process Creation Flags (WinBase.h) - Win32 - Microsoft Learn

This flag is ignored if the application is not a console application, or if it is used with either CREATE_NEW_CONSOLE or DETACHED_PROCESS.

Open-time Flags (The GNU C Library)

13.15.2 Open-time Flags. The open-time flags specify options affecting how open will behave. These options are not preserved once the file is open....

open(2) - Linux manual page - man7.org

The open() system call opens the file specified by pathname. If the specified file does not exist, it may optionally (if O_CREAT is...

Reading flags from a file - LaunchDarkly docs

This topic explains how to run feature flags from a file when you're using a server-side SDK.