Non-determinism in JSON export

Description of issue or feature request:

I’ve been working to create a deterministic test fixture generator for PHP-TUF. I’ve rooted out the apparent sources of most meaningful non-determinism by fixing the clock and using a fixed well of keypairs. However, some of the JSON export appears to have different behavior on different systems.

Shown below is the diff I see when comparing generated data on GitHub Actions (on Python 3.9 with ubuntu-latest) versus on my laptop (also Python 3.9 but with Fedora 33). We’ve pinned all known dependencies using pipenv, so I don’t think it’s that.

This causes a cascading set of differences because other files use hashes of snapshot.json.

Could TUF canonicalize even the JSON data that isn’t directly signed?

Current behavior:

Expected behavior:

Deterministic (ideally canonical) output of JSON that contains the same functional data.

[lukpueh]

Could TUF canonicalize even the JSON data that isn’t directly signed?

Note that python-TUF currently does not canonicalize any JSON metadata (not even the payload aka. “signed” part) on the wire, although there is a proposal to change this at least for the “signed” part to not require any JSON parsing of untrusted metadata see (https://github.com/secure-systems-lab/signing-spec/pull/2).

Canonicalization of the entire metadata is not required by the spec, because file hashes of targets.json, $delegated-targets.json (in snapshot.json) and snapshot.json (in timestamp.json) are generated and then re-generated for client verification over the same file blob (without need for JSON parsing/canonicalization).

Regardless, @erickt has made a similar request in https://github.com/theupdateframework/tuf/issues/1154 (in a similar context, i.e. interoperability testing).

I’m fine with implementing his suggestion.

[jku]

State as I understand it:

• We try to be deterministic in Metadata API: e.g. JSON dictionary content is sorted on output
• We do not canonicalize the output JSON (signed content is canonicalized but that is not used as the output format)

I’m closing this as it is about the legacy code which we no longer maintain: if you see a similar issue using Metadata API, please open a new issue