question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

error in to_pickle

See original GitHub issue

Hi, I think there is a bug in to_pickle

>>> data = pd.read_hdf(DG.load_path + 'time_series/PSI_TS_after_alignment_and_reshaping.h5', 'table')
>>> data.shape
(1006, 288095)
>>> data.to_pickle(DG.load_path + 'time_series/PSI_TS_after_alignment_and_reshaping2.pkl')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/donbeo/MyApps/phd_python/lib/python3.4/site-packages/pandas/core/generic.py", line 994, in to_pickle
    return to_pickle(self, path)
  File "/Users/donbeo/MyApps/phd_python/lib/python3.4/site-packages/pandas/io/pickle.py", line 14, in to_pickle
    pkl.dump(obj, f, protocol=pkl.HIGHEST_PROTOCOL)
OSError: [Errno 22] Invalid argument
>>> pd.version.version
'0.16.2'
>>> 

Issue Analytics

  • State:closed
  • Created 8 years ago
  • Comments:16 (11 by maintainers)

github_iconTop GitHub Comments

3reactions
shadziccommented, Oct 26, 2017

I had the same problem, the same error for pickling a large file. I solved the issue using this code:

import joblib
joblib.dump(df,'df_path.pkl')

So maybe a pandas’ issue?

2reactions
kawochencommented, Jan 28, 2016

probably not a pandas bug. check this out http://bugs.python.org/issue24658

Read more comments on GitHub >

github_iconTop Results From Across the Web

How to find source of error in Python Pickle on massive object
I'm trying to pickle this object, but I get this error: pickle.PicklingError: Can't pickle : it's not found as builtin.module.
Read more >
cannot run due to pickle error · Issue #88 - GitHub
Every time I try to run train.py I get the error TypeError: cannot pickle 'Environment' object as a result when trying to run...
Read more >
pickle — Python object serialization — Python 3.11.1 ...
The shelve module provides a simple interface to pickle and unpickle objects ... Error raised when there is a problem unpickling an object,...
Read more >
I cannot load the data due to pickle error – Q&A Hub | 365 Data ...
I cannot load the data due to pickle error. in SQL + Tableau + Python / Working with the "absenteeism_module". UnpicklingError Traceback (most...
Read more >
Pickle error when saving pandas dataframe - PyTorch Forums
Hi, I get this error: RuntimeError: Attempting to deserialize object ... get errors, however would prefer to pickle/ dump into an npy file....
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found