question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Make resources pickleable/serializable

See original GitHub issue

Boto3 resources (e.g. instances, s3 objects, etc.) are not pickleable and have no to_json() method or similar. Therefore, there’s currently no way to cache resources retrieved via boto3. This is problematic when retrieving a large number of resources that change infrequently. Even a cache of 30s or so can greatly increase the performance of certain programs and drastically reduce the number of necessary API calls to AWS.

Would it be possible to have some way to serialize resources?

Issue Analytics

  • State:open
  • Created 7 years ago
  • Reactions:28
  • Comments:18 (1 by maintainers)

github_iconTop GitHub Comments

20reactions
SoraDevincommented, Oct 12, 2020

Any update on this?

17reactions
iaineldercommented, Aug 17, 2021

This would make it much easier to use the multiprocessing library with boto3. For example, I would like to pass a session object and a list of organization accounts to the Pool.starmap function that calls a function that gets gets the tags on the account and merges them into the existing account objects.

Read more comments on GitHub >

github_iconTop Results From Across the Web

pickle — Python object serialization — Python 3.11.1 ...
Source code: Lib/pickle.py The pickle module implements binary protocols for serializing and de-serializing a Python object structure.
Read more >
The Python pickle Module: How to Persist Objects in Python
In the example above, you create several different objects and serialize them with pickle . This produces a single string with the serialized...
Read more >
Python serialization - Why pickle? - Stack Overflow
Pickling is a way to convert a python object (list, dict, etc.) into a character stream. The idea is that this character stream...
Read more >
Python Tutorial: serialization with pickle - 2020 - BogoToBogo
Python serialized object and JSON. The data format used by the pickle module is Python-specific. It makes no attempt to be compatible with...
Read more >
The ultimate guide to Python pickle - Snyk
Pickle can serialize a Python object into a flat byte stream (pickling) ... of our newly created state.bin file, it won't make much...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found