Strange behavior from Metadata.sign() and Metadata.to_file()

Description of issue or feature request: I made some tests locally to answer the question: do different orders in the dictionaries in the metadata objects lead to a different signature?

I wrote this small script aiming to change the order of the Targets.target dictionary content as the insertion order is preserved.

from tuf.api.metadata import Metadata, Targets
from securesystemslib.signer import SSlibSigner
from securesystemslib.interface import import_ed25519_privatekey_from_file
from tuf.api.serialization.json import JSONSerializer
import copy

serializer = JSONSerializer(compact=False)
old_target = Metadata.from_file("<LOCAL_PATH_TO_TUF>/tests/repository_data/repository/metadata/targets.json")

targets_key = import_ed25519_privatekey_from_file("<LOCAL_PATH_TO_TUF>/tests/repository_data/keystore/targets_key", password="password")
targets_signer = SSlibSigner(targets_key)

new_target = copy.deepcopy(old_target)
if isinstance(new_target.signed, Targets) and isinstance(old_target.signed, Targets):
    new_target.signed.targets = {}
    new_target.signed.targets["file2.txt"] = old_target.signed.targets["file2.txt"]
    new_target.signed.targets["file1.txt"] = old_target.signed.targets["file1.txt"]
    print(new_target == old_target)
    new_target.sign(targets_signer)
    new_target.to_file("targets_new_file.json", serializer)


for key, val in new_target.signed.targets.items():
    print(f"{key}: {val}")

and I expected that I will get one of the two scenarios:

1. the targets_new_file.json will have the same signature as the original targets.json, but a different order in the singed.targets dictionary
2. the targets_new_file.json will have a different signature compared to the original targets.json and different order in the singed.targets dictionary

Instead, I got a third scenario: the targets_new_file.json has the same signature as the original targets.json and the SAME order in the singed.targets dictionary. The terminal output looks like this:

True
file2.txt: <tuf.api.metadata.TargetFile object at 0x7f588e1c0070>
file1.txt: <tuf.api.metadata.TargetFile object at 0x7f588e21b7c0>

meaning the insertion order is indeed preserved during iteration.

I debug it locally and I found the following:

1. We have the following calls: new_target.sign(targets_signer) -> CanonicalJSONSerializer.serialize() -> securesystemslib.formats.encode_canonical() I noticed a problem inside securesystemslib.formats.encode_canonical(). https://github.com/secure-systems-lab/securesystemslib/blob/602d3fcd59f726fc64e78a93430f427712d26ca4/securesystemslib/formats.py#L646. The problem is that no matter the dict order at line 646 inside securesystemslib.formats we are sorting it. In my case I always observed that new_targets.signed_targets containing file2 and file1 was traversed as file1 and then file2. Should we fix securesystemslib.formats.encode_canonical()? This means that signing two files with the same content, but a different order of delegations will receive the same signature. Is that okay?
2. Not sure how that happens, but I see the same problem as 1) when we call Metadata.to_file. No matter the new_targets.targets order I always saw file1 was first and file2 was second. This time there is no securesystemslib call that made me suspicious, but instead, the problem should be somewhere inside Metadata.to_bytes(). While debugging I found that it’s not the problem when calling md_dict = metadata_obj.to_dict() inside JSONSerializer.serialize() as the md_dict has the same order of the targets as the new_targets object. So, I am not sure where is the problem here…

@jku or @lukpueh does anyone from you have an opinion? I am sure this is hard to follow, but I didn’t find a better way to put it and I didn’t want to forget my findings.

[jku]

Yes, I agree with that list.

So I think this issue can be closed? Martin, let us know if you think there’s something not covered above (or if the documentation needs to be improved)

[jku]

Great testing, we should be doing more of this sort of verification of assumptions.

My first thought is that you may have mixed delegated roles and targets (a mistake I’ve made before) in that within Targets metadata, former is ordered but the latter is not: targets order does not matter but delegated roles does.

I’m not totally sure why the targets dict order you chose isn’t reflected in the serialized format… but encode_canonical() seems to work as expected: dicts like “targets” are sorted (because their order should not affect canonicalization) but lists like “delegated roles” are not sorted because the order matters.