question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Different HDFStores in multiple threads crashes Python

See original GitHub issue
import threading
import pandas as pd
import time

def foo():
    store = pd.HDFStore('my_hdf_file.h5')
    store['foo']
    store.close()


def main():
    threading.Thread(target=foo).start()
    threading.Thread(target=foo).start()
    time.sleep(2)

if __name__ == '__main__':
    main()

Crashes for me (Windows 7 using pytables 2.4.0 and pandas 0.9.1 from http://www.lfd.uci.edu/~gohlke/pythonlibs/). I can’t get the stack trace easily, I can try harder if necessary. Simply using tables.openFile and reading a few values seems to work fine.

Issue Analytics

  • State:closed
  • Created 11 years ago
  • Comments:6 (5 by maintainers)

github_iconTop GitHub Comments

1reaction
wesmcommented, Dec 6, 2012

We could potentially add locks to HDFStore at some point to prevent multiple threads from accessing the file at once

Read more comments on GitHub >

github_iconTop Results From Across the Web

pytables crash with threads - python - Stack Overflow
Use a dedicated process that handles all the hdf5 I/O. Processes/threads of the web application must communicate with this process through, ...
Read more >
How to Tell if Too Many Threads Are Crashing My ...
How can I tell if my multithreaded Python program is crashing on my Pi 3 because there are too many threads? (Or poorly...
Read more >
48557 (py27-pandas crashes with blosc) - MacPorts Trac
I am trying to create an HDF5 file, and if I use blosc compression it segmentation faults every time when I try to...
Read more >
Issue 42969: pthread_exit & PyThread_exit_thread from ...
msg385288 ‑ (view) Author: Gregory P. Smith (gregory.p.smith) * Date: 2021‑01‑19 19:35 msg385289 ‑ (view) Author: Gregory P. Smith (gregory.p.smith) * Date: 2021‑01‑19 19:58 msg396669...
Read more >
Segfault and crash embedding when julia runs multithreaded gc
I'd assume nothing about program behavior after races. Out of curiosity: do you have two different garbage collectors running (Julia and ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found