Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Use low level API to create h5py objects from ID

See original GitHub issue

Hi,

I’m trying to write python wrapper for HighFive C++ hdf5 wrapper via pybind11. On the lower level I would like to make it possible creating h5py objects from HighFive object’s ID (and vice versa).

I create HighFive::Group in python and then I try to create h5py group from HighFive::Group’s ID (see inline comments):

import HFGt   # it is HighFive fork
import h5py

hfgt_file = HFGt.File("myFile.h5", HFGt.OpenFlag.Overwrite)
hfgt_file_id = hfgt_file.getId(False)
hfgt_group = hfgt_file.createGroup("myGroup", HFGt.LinkCreateProps(), HFGt.GroupCreateProps(), HFGt.GroupAccessProps())
hfgt_group_id = hfgt_group.getId(False)  # 144115188075855872

h5py_file = h5py.File('myFile_py.h5','w')
h5py_file_id = h5py_file._id
h5py_group = h5py_file.create_group("myGroup")
h5py_group_id = h5py_group._id  # 144115188075855873

# here I try to instantiate h5py::Group from HighFive::Group's ID
h5py_group = h5py.Group(hfgt_group_id)  # Exception has occurred: ValueError. 144115188075855872 is not a GroupID

I use the same HDF5 C-lib that h5py uses. As you can see hfgt_group_id is 144115188075855872, wich is by one smaller than h5py_group_id and this proves that both libs uses the same HDF5 lib.

How do you think, why I get this error and I can’t instantiate h5py::Group from HighFive::Group's ID ?

My configuration:

Operating System: Windows 10
Python version: 3.8.5
Where Python was acquired: Windows miniconda
h5py version: 2.10.0
HDF5 version: 1.10.4
The full traceback/stack trace shown (if it appears)

Issue Analytics

State:
Created 3 years ago
Comments:6 (3 by maintainers)

Top GitHub Comments

1reaction

kerim371commented, Feb 22, 2021

@takluyver thank you very much!

0reactions

takluyvercommented, Feb 22, 2021

Yes, that’s right. The Cython code for initialising one of these low-level objects is here, and you can see nothing is incrementing a refcount (the specific FileID, GroupID etc. classes subclass this ObjectID class - you can verify if you like that they’re not incrementing the ref count either):

https://github.com/h5py/h5py/blob/64f7b7da5e996d99ab29c64e7bb95e5a13a0c850/h5py/_objects.pyx#L184-L191

For comparison, here’s the code called when an object is released, which calls H5Idec_ref:

https://github.com/h5py/h5py/blob/64f7b7da5e996d99ab29c64e7bb95e5a13a0c850/h5py/_objects.pyx#L194-L206

Top Results From Across the Web

Low-Level API — h5py 3.7.0 documentation

This documentation mostly describes the h5py high-level API, which offers the main features of HDF5 in an interface modelled on dictionaries and NumPy...

Module H5D — Low-level API for h5py (v3.7.0)

Represents an HDF5 dataset identifier. Objects of this class may be used in any HDF5 function which expects a dataset identifier. Also, all...

h5py Low-Level API Reference

This documentation contains the auto-generated API information for the h5py 3.7.0 “low-level” interface, a collection of Cython modules which form the ...

File Objects — h5py 3.7.0 documentation

HDF5 handles filenames as bytes (C char * ), and the h5py Low-Level API matches this. macOS (OSX)¶. macOS is the simplest system...

Module H5I — Low-level API for h5py (v3.7.0)

Determine the HDF5 typecode of an arbitrary HDF5 object. The return value is always one of the type constants defined in this module;...