question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

botocore exceptions are not picklable

See original GitHub issue

they need to take a “msg” argument, and all other arguments must be optional:

http://stackoverflow.com/questions/16244923/how-to-make-a-custom-exception-class-with-multiple-init-args-pickleable. I’ve attempted fixing this in PR: https://github.com/boto/botocore/pull/835

simple testcase:

try:
    raise botocore.exceptions.NoCredentialsError
except Exception as e:
    b = pickle.dumps(e)
    e = pickle.loads(b)

Issue Analytics

  • State:closed
  • Created 8 years ago
  • Reactions:2
  • Comments:6 (1 by maintainers)

github_iconTop GitHub Comments

3reactions
jezdezcommented, May 4, 2017

@thehesiod I’ve opened #1191 that uses __reduce__ and which also tackles #899 to have a consistent exception error set. Bonus feature is being able to import the exception classes under botocore.exceptions.<service>.<exceptioncls>, e.g. as botocore.exceptions.s3.BucketAlreadyExists

2reactions
thehesiodcommented, Mar 8, 2017

this is now further compounded by the metaclass instantiation of ClientError, which return things like botocore.errorfactory.NoSuchKey, which can’t pickle because that class does not exist in that module. I think a better impl is probably implementing the __reduce__ methods

Read more comments on GitHub >

github_iconTop Results From Across the Web

Python multiprocessing PicklingError: Can't pickle <type ...
SimpleQueue must be pickable, and foo.work is not picklable since it is not defined at the top level of the module. It can...
Read more >
Better handling of errors that are not pickleable when testing ...
One example of error like that is botocore.exceptions.ClientError which pickles as "ClientError(message)", but its initializer is actually ClientError(response, ...
Read more >
pickle — Python object serialization — Python 3.11.1 ...
Note that other exceptions may also be raised during unpickling, including (but not necessarily limited to) AttributeError, EOFError, ImportError, and ...
Read more >
Python Tutorial: serialization with pickle - 2020 - BogoToBogo
Not every Python data structure can be serialized by the pickle module. ... 0: raise Exception(stderr) else: if data: procs_data.append(data) return stdout ...
Read more >
Persisting Data with Pickle & S3 - datawookie
If it's not there, handle the error gracefully. python. try: s3.download_file(BUCKET, PICKLE, PICKLE) except botocore.exceptions.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found