question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Support objects with buffer-protocol as an input to loads

See original GitHub issue

Currently the input to orjson.loads must be a bytearray or a bytes object. It would be useful if possible to also support passing in other objects supporting the buffer protocol.

In my use case, I have a large bytearray that contains some serialized json at known offsets. If possible, I’d like to read this in without having to copy it out of the bytearray into a new bytearray that only contains the json. Example:

In [25]: header = b"some bytes before"

In [26]: footer = b"some bytes after"

In [27]: json = orjson.dumps({"key": "val"})

In [28]: data = bytearray(header + json + footer)  # a potentially large bytearray

In [29]: subset = memoryview(data)[len(header):len(header) + len(json)]  # a view on the json data, without a copy

In [30]: bytearray(subset)  # this makes a copy, would be nice to just pass `subset` to loads
Out[30]: bytearray(b'{"key":"val"}')

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Reactions:2
  • Comments:5 (3 by maintainers)

github_iconTop GitHub Comments

3reactions
ijlcommented, Feb 24, 2021
1reaction
ijlcommented, Jul 24, 2020

@mattsta thanks for the research, it’s helpful.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Overview | Protocol Buffers - Google Developers
Protocol buffers support many scalar value types, including integers that use both variable-length encoding and fixed sizes. You can also create ...
Read more >
Buffer Protocol — Python 3.11.1 documentation
Buffer structures (or simply “buffers”) are useful as a way to expose the binary data from another object to the Python programmer. They...
Read more >
memoryview() in Python - GeeksforGeeks
Buffer protocol provides a way to access the internal data of an object. This internal data is a memory array or a buffer....
Read more >
json - ESP32: Odd behavior with urequests - object with buffer ...
ESP32: Odd behavior with urequests - object with buffer protocol required ... ESP32 module with ESP32 Type "help()" for more information.
Read more >
Glossary — MicroPython 1.19 documentation - OpenMV Docs
buffer protocol ¶. Any Python object that can be automatically converted into bytes, such as bytes , bytearray , memoryview and str objects, ......
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found