Updater feature request: verify chain of trust from bootstrapped root metadata

Description of issue or feature request:

Context:

• In order to comply with the detailed client workflow a TUF client must ship a bootstrap trusted root metadata file out-of band (i.e. with the client installer).

Observations:

• On general purpose operating systems (Linux, Windows, macOS) this bootstrap root metadata will often be installed into operating system owned storage (i.e. the Python that comes with macOS, a distro installed Python on Linux, or Python from the Microsoft Store on Windows).
• In the case of pip, and possibly other integrations, the trusted root metadata downloaded and stored during the detailed client workflow will often be persisted in user owned storage (i.e somewhere in a UNIX users home directory).
• In many cases the bootstrap root metadata will be afforded a much higher level or protection from tampering (SELinux/MAC, integrity verification with IMA/EVM, etc on some Linux distros. System integrity protection for the default Python install on recent macOS.)

With these considerations in mind, it feels like providing a way to verify the chain of trust from the boostrap root metadata to the trusted root metadata will provide some additional integrity protection for the client, to better detect tampering with trusted root metadata that isn’t stored in OS protected locations.

Current behavior:

No verification of chain of trust from bootstrap trusted root metadata to current trusted root metadata.

Expected behavior:

Expose an optional method on tuf.client.updater.Updater, to be called before refresh(), which will walk the chain of trust from the bootstrap root metadata to the current trusted root metadata. This method will perform a similar sequence of steps as defined in 5.1 of the detailed client workflow, only it will prefer local copies of the intermediate trusted root metadata before reaching out to the repository to download intermediate root metadata file version.

[jku]

The original issue (chaining root trust from bootstrap root.json) should now be fairly easy to implement in ngclient:

• Updater should take the bootstrap root file (or bytes?) as optional constructor argument, and initialize the TrustedMetadataSet with it
• Updater should be modified to store versioned valid root metadata files (<N>.root.json) in the local metadata store. Even if Updater was not itself bootstrapped, it should store versioned root files. maybe also any time a versioned root is stored, non-versioned root.json(symlink?) should be stored so non-bootstrap updater knows what to start with.
• When loading root metadata if a bootstrap root.json is given, Updater should load local metadata versions (starting from bootstrap version onwards) first, and continue with remote if the local one is not available or valid. So root update would look like:
  • initialization:
    • if boostrapped: Updater gets bootstrap root vN as constructor argument, loads into TrustedMetadataSet, stores versioned copy in cache
    • if not bootstrapped: Updater reads non-versioned root from local cache, loads into TrustedMetadataSet (this is what currently happens)
  • Updater root update loop continues as long as root versions are found:
    • if valid root N+1 found in local cache, load into TrustedMetadataSet
    • elif valid root N+1 found on remote server, load into TrustedMetadataSet, store a versioned copy in cache
    • else consider N the final root, continue as usual to timestamp update

[mnm678]

Another thing I had not considered before: we may want to provide multiple bootstrap files (targets.json and bins.json as well as root.json). This is not really useful for verifying the chain of trust of an existing runtime metadata but the point would be to make the first run pip invocation faster: the client would not need to download the large bins.json file (assuming the included file has not expired).

This would also provide some additional rollback protection for first time clients as they could use bootstrapped targets and maybe snapshot to ensure version numbers have not decreased. The added metadata may make the initial install much larger, but it might be useful in some cases.