question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

PR #77 returns raw bytes as output, resulting output that is not parsable as json in current(0.19) release when using python 3

See original GitHub issue

When using 0.19 awscurl with python 3, encoding response.text results in a byte object being returned. Printing that directly results in the output being prefixed with b' (see screenshots below), and thus json output cannot be parsed directly from the response. In Python 2 this is not an issue. I could also see this being an issue for (say) an xml based response as well (though I have not tested that).

My use case: I’m currently invoking awscurl in a unix pipe, where the result of awscurl is piped to jq for further processing (in CI); updating to the latest version broke the pipe. For the moment I’ve pinned the version in CI at 0.17

Python 3 screenshot

Screen Shot 2019-12-18 at 4 50 09 PM

Python 2 screenshot

Screen Shot 2019-12-18 at 4 50 47 PM

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Reactions:11
  • Comments:17 (8 by maintainers)

github_iconTop GitHub Comments

2reactions
p5k6commented, Oct 21, 2020

Fixed in version 0.21 - thank you @okigan!

0reactions
p5k6commented, Oct 21, 2020

FYI - I’ve added a PR (#92) to address this issue, including a unit test which addresses this issue.

Note that - the response should have the info needed re: the encoding via the api request (response.encoding should be set per the data included in the headers in the response object). By adding .encode(), awscurl forces us to have a bytes output, instead of relying upon how python would natively interpret the encoding provided by the response encoding.

I believe that my PR includes the correct way to handle this issue. The issue raised in #77 was specific to python 2, which is no longer maintained (in fact, I couldn’t get a development environment using python 2 to spin up).

Read more comments on GitHub >

github_iconTop Results From Across the Web

Help: json.loads() cannot parse valid json - Python Help
string and are visible to the JSON encoder, or use a raw string. Tested and works: You can fix the escape sequence problem...
Read more >
Why can't Python parse this JSON data? - Stack Overflow
Your data is not valid JSON format. You have [] when you should have {} for the "masks" and "parameters" elements: [] are...
Read more >
Read, Write and Parse JSON using Python - GeeksforGeeks
Parse JSON (Convert from JSON to Python). json.loads() method can parse a json string and the result will be a Python dictionary. Syntax:...
Read more >
Extract scalar values from JSON data using JSON_VALUE()
This article gives an overview of JSON_VALUE() function to extract scalar value from the JSON Data.
Read more >
JSON methods, toJSON - The Modern JavaScript Tutorial
Let's say we have a complex object, and we'd like to convert it into a string, to send it over a network, or...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found