Large numbers in JSON will not be represented correctly in JavaScript

Large integers, like 9999999999999999999999999 are valid in JSON files and are valid as data parts in the 0xcert conventions. However these numbers are currently not representable in JavaScript in full fidelity. This violates some guarantees made by (or assumptions you might make while using) the 0xcert Framework. Workarounds are available.

Discussion

These examples are running Safari Version 12.0.3 (14606.4.5).

Large numbers cannot currently be represented as integer literals.

9999999999999999999999999 == 9999999999999999999999998
// Outputs: true

However, they can be represented in JSON on your server or some other environment. If we find two such file, we can compare the JSON string.

a = '{"tokenId": 9999999999999999999999999}';
b = '{"tokenId": 9999999999999999999999998}';
a == b;
// Outputs: false

But if you decode those objects and compare…

a = '{"tokenId": 9999999999999999999999999}';
b = '{"tokenId": 9999999999999999999999998}';
JSON.stringify(JSON.parse(a)) == JSON.stringify(JSON.parse(b))
// Outputs: true

The 0xcert Framework currently uses native JSON parsing. This means…

⚠️ If 0xcert Framework is acting on a JSON that includes integers larger than the JavaScript safe limit then you will not have access to the full fidelity of those integers.

And also…

⚠️ If 0xcert Framework is testing/exposing unsafe large integers then the test/proof results might be inaccurate.

Workaround

If you will use large integers encoded as strings in your JSON then this does not affect you.

Example:

{
  "tokenId": "4045927345087450934570349857"
}

Also, if you will use the same environment running 0xcert Framework to create JSON files as is used to consume the files, then this does not affect you. Or in other words, in this case it is a JavaScript problem and you are programming in JavaScript when sending data to 0xcert Framework, therefore you should already know that you cannot send the number 9999999999999999999999999 as an integer to the 0xcert Framework with full fidelity.

Work plan

Steps for now

• Document this known issue in the relevant places
  • Conventions documentation
  • Certification module documentation
  • Conceptual documentation
  • Release notes

Long-term plan

• Upgrade to safer underlying JavaScript
  • Track solutions (BigNum, TypeScript polyfill?) and their deployment status
  • Require newer version of JavaScript for this Framework, or require the use of Babel or similar crosscospiler

-or-

• Implement safe JSON parse function (possibly a separate NPM)
• Require this safe JSON parser

References

• 0xcert Framework issue #336
• https://github.com/douglascrockford/JSON-js/issues/107#

[sbelak]

Feels like a warning should be emitted somewhere if number is an int and > MAX_SAFE_INTEGER.

[alko89]

I just found this relevant reference: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Number/MAX_SAFE_INTEGER

Basically it states that double-precision format can only safely represent numbers between -(2^53 - 1) and 2^53 - 1. The same format applies to numbers inside a JSON structure and is a design limitation of double-precision numbers.

Personally I think this should be addressed on the language specification level (which is not in case of JavaScript) or on the end user application. Having this limitation within the framework could potentially ruin a use case.