question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

ujson cannot handle long values correctly

See original GitHub issue

I noticed that when getting numeric values out of a JSON, you can only retrieve the number as a double using the num method. So if I know that I’m dealing with an 64-bit integer, my only option is to do this:

val id = json("some_64_bit_integer").num.toLong

(Also tried .string.toLong, but that resulted in a runtime error because numeric values cannot be queried as strings)

The problem is that this only works reliably up to 53-bit integers, above that we’ll silently lose precision! Meaning that there will be no indication of any error whatsoever, but values greater than 2^53 will be just wrong!

One possible solution to this would be to have different methods for all numeric Java datatypes, so byte, short, int, long, float and double instead of just num. We could still use the wrong method and then precision would be potentially lost, but at least then we’d have the option to get the correct values out if we used the correct method.

Additionally, in theory numeric JSON values can represent big numbers as well that cannot fit into the signed 64-bit range of a Java long. So ideally we’d have a bigDecimal method as well.

Check this for a more detailed explanation on the issue (it’s a JavaScript related writeup, but don’t let that confuse you, they are apparently facing the same precision problems in JS-land because they only have doubles natively 😃).

http://2ality.com/2012/07/large-integers.html

Issue Analytics

  • State:closed
  • Created 5 years ago
  • Comments:5 (3 by maintainers)

github_iconTop GitHub Comments

2reactions
johnnovakcommented, Jan 18, 2021

But JavaScript is shit and we should do better, IMHO 😎

1reaction
lihaoyicommented, Jan 14, 2021

@radekm that behavior is expected. ujson.Value follows Javascript semantics, which behaves as shown on large numbers. You can use upickle.default.Read[Map[String, Long]] for your example, or use a different AST (https://www.lihaoyi.com/upickle/#OtherASTs) if you want a more precise AST structure (at the expense of performance and other things).

Read more comments on GitHub >

github_iconTop Results From Across the Web

ujson cannot handle long values correctly · Issue #263 - GitHub
I noticed that when getting numeric values out of a JSON, you can only retrieve the number as a double using the num...
Read more >
Minimal Python object that can't be uJson'd - Stack Overflow
I'd like to write a unit test that shows that we correctly handle ... Hm. ujson seems to try very hard to produce...
Read more >
Inserted a json got $numberLong back in the returned json
With this solution, I'm sure the JSON is actually legit and it's your client's problem to deal with the parsing of this value...
Read more >
lihaoyi/upickle - Gitter
I have a problem using upickle with type aliases in a cake pattern. Having a definition like this: @ trait QM { type...
Read more >
Json vs. simplejson vs. ujson - Hacker News
cjson's way of handling unicode is just plain wrong: it uses utf-8 bytes as unicode code points. ujson cannot handle large numbers (somewhat ......
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found