botocore exceptions are not picklable
See original GitHub issuethey need to take a “msg” argument, and all other arguments must be optional:
http://stackoverflow.com/questions/16244923/how-to-make-a-custom-exception-class-with-multiple-init-args-pickleable. I’ve attempted fixing this in PR: https://github.com/boto/botocore/pull/835
simple testcase:
try:
raise botocore.exceptions.NoCredentialsError
except Exception as e:
b = pickle.dumps(e)
e = pickle.loads(b)
Issue Analytics
- State:
- Created 8 years ago
- Reactions:2
- Comments:6 (1 by maintainers)
Top Results From Across the Web
Python multiprocessing PicklingError: Can't pickle <type ...
SimpleQueue must be pickable, and foo.work is not picklable since it is not defined at the top level of the module. It can...
Read more >Better handling of errors that are not pickleable when testing ...
One example of error like that is botocore.exceptions.ClientError which pickles as "ClientError(message)", but its initializer is actually ClientError(response, ...
Read more >pickle — Python object serialization — Python 3.11.1 ...
Note that other exceptions may also be raised during unpickling, including (but not necessarily limited to) AttributeError, EOFError, ImportError, and ...
Read more >Python Tutorial: serialization with pickle - 2020 - BogoToBogo
Not every Python data structure can be serialized by the pickle module. ... 0: raise Exception(stderr) else: if data: procs_data.append(data) return stdout ...
Read more >Persisting Data with Pickle & S3 - datawookie
If it's not there, handle the error gracefully. python. try: s3.download_file(BUCKET, PICKLE, PICKLE) except botocore.exceptions.
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found

@thehesiod I’ve opened #1191 that uses
__reduce__and which also tackles #899 to have a consistent exception error set. Bonus feature is being able to import the exception classes underbotocore.exceptions.<service>.<exceptioncls>, e.g. asbotocore.exceptions.s3.BucketAlreadyExiststhis is now further compounded by the metaclass instantiation of ClientError, which return things like botocore.errorfactory.NoSuchKey, which can’t pickle because that class does not exist in that module. I think a better impl is probably implementing the
__reduce__methods