question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Create a dataset if it does not already exist

See original GitHub issue

I am trying to create a dataset if it does not exist with the following statement

a=file.get("ds", file.create_dataset("ds", shape=(0, 1), maxshape=(None, 1), data=None,chunks=True))

But I am getting the error RuntimeError: Unable to create link (name already exists) when the dataset ds already exists, which probably means that file.create_dataset("ds", shape=(0, 1), maxshape=(None, 1), data=None,chunks=True) is called when the dataset already exists, even though it should not be called. A workaround would be to

a=file.get("ds")
if a is None:
    a=file.create_dataset("ds", shape=(0, 1), maxshape=(None, 1), data=None,chunks=True)

However, it seems like file.get should handle this situation.

Here are my specifications

  • Operating System: Mac OS Mojave (10.14.6)
  • Python version: 2.7.15+
  • Where Python was acquired: system Python
  • h5py version: 2.7.1 (h5py.__version__)
  • hdf5 version: 1.10.0
  • The full traceback/stack trace shown (if it appears)
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "/usr/lib/python2.7/dist-packages/h5py/_hl/group.py", line 109, in create_dataset
    self[name] = dset
  File "/usr/lib/python2.7/dist-packages/h5py/_hl/group.py", line 277, in __setitem__
    h5o.link(obj.id, self.id, name, lcpl=lcpl, lapl=self._lapl)
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper (/build/h5py-qzs83i/h5py-2.7.1/h5py/_objects.c:2847)
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper (/build/h5py-qzs83i/h5py-2.7.1/h5py/_objects.c:2805)
  File "h5py/h5o.pyx", line 202, in h5py.h5o.link (/build/h5py-qzs83i/h5py-2.7.1/h5py/h5o.c:3899)
RuntimeError: Unable to create link (name already exists)

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Comments:8 (3 by maintainers)

github_iconTop GitHub Comments

1reaction
takluyvercommented, Sep 30, 2019

My previous comment was meant to be a polite way of saying: please work it out yourself.

0reactions
AbdulKaderJaleelcommented, Dec 16, 2021

here is the sample code:

with h5py.File(‘filtered.h5’, ‘a’) as f: # Create a group called data print(f.keys()) if f.keys()== None: g = f.create_group(‘data’) g.create_dataset(filename, data=norm_image) else: g = f.require_group(‘data’) g.create_dataset(filename, data=norm_image)

Read more comments on GitHub >

github_iconTop Results From Across the Web

JCL to create dataset whether it already exists or not
If what you want is to create the dataset if it doesn't already exist and append to it if it does, that can...
Read more >
Solved: Create a table, but only if it doesn't exist already
I have some code that creates an empty table, then loads it with data from external sources. The program is meant to be...
Read more >
How to INSERT If Row Does Not Exist (UPSERT) in MySQL
The obvious purpose is to execute a large number of INSERT statements for a combination of data that is both already existing in...
Read more >
Creating datasets | BigQuery - Google Cloud
For Dataset ID, enter a unique dataset name. · For Data location, choose a geographic location for the dataset. After a dataset is...
Read more >
Why Testing for Data Set Existence with the EXIST Function ...
Test (if it didn't already exist), while the code performs no action the second time the loop is iterated, as demonstrated in the...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found