External Signing API, or CCID/PIV Support

Purpose

It would be good to make it easier to manage repository metadata with TUF’s repository_tool module while using specialized hardware signing mechanisms that do not pass private keys around in memory. Currently, repository_tool expects to be provided private keys that it can use to produce signatures.

Current workaround

The workaround currently involves producing metadata normally (with unchanged TUF code) through repository_tool using dummy keys, and then using external code to, without TUF, replace signatures in the produced JSON metadata, for any signatures to be made with private key data that cannot be provided to TUF. These can just be swapped into the JSON metadata that repository_tool produces. This should be easy. (Caveat: If Snapshot, specifically, has to be signed with an external key, then in addition to replacing that signature, one must also re-calculate Timestamp’s hash of snapshot metadata and re-sign Timestamp. One could make TUF calls to do the Timestamp adjustment after externally re-signing Snapshot in that case.)

Options for the Future

Option 1: External Signing Support

A PR has been put forward to suggest an API defining key signers via an extension of the metadata combined with additional handler mappings for all key types, where the mapped-to value is a class that supports a signature-related functions. I think this is unfortunately a little over-engineered for the codebase of the reference implementation, requiring additional classes and metadata changes, and impairing readability. 😔

I have to do some thinking about what a good API will look like if we go the direction of adding general external signing support. I’d prefer something much thinner and less intrusive. Off the top of my head, while I don’t love it, here’s an example:

• a (dummy if necessary) key is provided that lists a scheme value identifying the (internally) unsupported signing scheme (or some ‘custom’ value)
• loading the key into TUF is permitted despite the unknown signing scheme (or, permitted if the scheme is ‘custom’, say)
• when signing with the “key”, TUF notes the scheme is unsupported (or perhaps an expected value like ‘custom’ or ‘external’) and calls out to an external handler module with a fixed name, providing the bytes to be signed and the (potentially dummy) “key”, taking the result as the signature.
• the external module can be provided by a separate dependency and need only support a one-function interface producing a signature through whatever mechanism is appropriate for the key as determined externally based on scheme.

I’m very much open to alternative suggestions. (@lukpueh?)

Option 2: TUF support for CCID / PIV

There also may be additional (to OLL) interest in this. @trishankatdatadog? It seems to me like it would be reasonable to add some optional dependencies (similar to the PyNaCl and cryptography libraries) that could handle this and would require some non-Python dependencies. It’s likely that the API for this would look like dummy private keys, similar to what I suggested above. I’m curious, though, about how in-toto handles this. (Indirectly via gpg…?) @lukpueh @SantiagoTorres

I also have some reading to do about functionality for CCID/PIV support in Notary.

[lukpueh]

@awwad and I talked this through and agreed that an internal solution that allows external signing as part of the whole TUF metadata signing process (see repository_tool.writeall) is preferable. The other approach of restructuring things, so that the management code has the freedom to choose whatever signing process it wishes, puts too much responsibility on the management code to sign the right thing at the right time (also Seb’s https://github.com/theupdateframework/tuf/issues/864#issuecomment-487681205 above for the two approaches).

@danixeee’s proposition in openlawlibrary/tuf#6 and is actually pretty much what we want. It extends keydb to also store signature provider functions that can be individually defined by the management code as long as they take the data to be signed as argument and return a signature in the desired TUF format. repository_lib.sign_metadata, called in writeall, may then use the passed signing keyids to fetch either a signing private key from keydb and provide the signature itself (status quo) or pass the data to the signature provider function, also stored at a keyid in keydb, and add the returned signature.

If a keyid in the keydb holds both a signing key and a signature provider function, prioritization may be resolved by sign_metadata. But keydb could also allow only one of private key or signature provider function for a given keyid.

On a more general note, I saw that keydb does not differentiate between public and private keys. It is up to the caller to check if a queried signing key does have the private portion, or to not accidentally disclose a private key, when querying an assumed public key. It might make sense to better separate public and private keys in keydb to make such accidents less likely. Note that in the case of signature provider functions, public keys need to be stored separately anyway. What do others think?

@JustinCappos, can you point me to resources about the partial signatures discussion.

[trishankatdatadog]

Yeah, I’m not sure we want to expose intermediate dummy keys, that doesn’t sound like the right abstraction

The right abstraction is: here’s a piece of metadata ready to be signed by this pubkey / keyid, something implementing a shared interface signs it, then the caller (presumably TUF repo tool) optionally checks whether the signature matches the pubkey and the message

Does this roughly make sense?