Support custom signing implementations in Metadata.sign method

Description of issue or feature request: Allow TUF integrators to easily use custom metadata signing implementations instead of the currently supported securesystemslib one.

An urgent use case for this is the PEP458/Warehouse integration, where metadata must be signed with keys stored in a Hashicorp Vault and the implementation is already available.

NOTE: This feature request only concerns the tuf.api.Metadata.sign method and not the legacy signing facilities write and writeall in repository tool.

Current behavior: tuf.api.Metadata.sign(key, ...) calls securesystemslib.keys.create_signature with the passed sslib/json-formatted private key.

Expected behavior: tuf.api.Metadata.sign takes an abstract signer parameter, which implements signing. A default signer may resolve to the currently used securesystemslib.keys.create_signature.

Implementation considerations:

• Different signers require completely different “keys”, e.g. the currently used generic implementation requires an sslib/json-formatted private key, whereas signing implementations that don’t have direct access to the key may only need a keyid (see GPG or HSM references below). As a consequence the key parameter in Metadata.sign should probably be encapsulated within the signer.

• The abstract signer interface must strictly define the format of the returned signature for metadata interoperability.

• The abstract signer interface design should also keep in mind other key-related tasks, such as signature verification, public key export to sslib metadata format, and keyid generation.

• securesystemslib.keys.create_signature is already an abstract interface, where the json-formatted private key parameter format is generic enough to hold different key types and to choose one of many supported signing algorithms, identified by the key’s scheme field. (see public key metadata format reference below). However, it is not suited to extend to custom implementations. A new abstract signer interface should integrate well with the existing securesystemslib infrastructure and/or aim to replace it. It should not become yet another co-existing interface and/or layer of abstraction (see public API overview reference below).

---

Related work and references: (for the very ambitious reader)

• securesystemslib key overview (to coordinate this feature request with the existing infrastructure)

  • general public API overview – sslib#270
  • generic signing interface currently used by TUF – sslib.keys.create_signature
  • GPG signing interface – sslib.gpg.functions.create_signature
  • WIP: HSM signing interface – sslib.hsm.create_signature
  • general public key metadata format – sslib#308
  • in-memory key representation for generic signing interface – sslib#310
  • stand-alone on-disk key format for generic signing interface – sslib#309

• Recently added abstract filesystem interface (for design inspiration) – https://github.com/theupdateframework/tuf/issues/1009

• @trishankatdatadog’s Hashicorp Vault interface POC (as reference for PEP458/warehouse implementation and for design inspiration) – VaultKey

• Abstract pyca/cryptograhpy key interface with sign method (for design inspiration) – RSAPrivateKey

• “Historical” external signing API feature request (has some interesting discussion and designs but targets legacy TUF codebase) – https://github.com/theupdateframework/tuf/issues/864

• in-toto metadata model sign methods (alternative but less flexible approach to support different signing implementations)

  • implementation – Metadata.sign*
  • related design discussion (replace serialization/deserialization with sign/verify!) – sslib#272

[lukpueh]

A very basic implementation could look something like this:

# in securesystemslib
class Signer:
    @abc.abstractmethod
    def sign(bytes: payload) -> Signature:
        raise NotImplementedError

class SSlibSigner(Signer):
    def __init__(self, sslib_private_key):
        self.key = sslib_private_key

    def sign(self, payload):
        return sslib_keys.create_signature(self.key, payload)

class GPGSigner(Signer):
    def __init__(self, gpg_keyid):
        self.keyid = gpg_keyid

    def sign(self, payload):
        return sslib_gpg.create_signature(payload, self.keyid)

# in TUF
class Metadata:
    def sign(sslib.Signer: signer):
        signer.sign(self.signed.to_canonical_bytes())

# in Warehouse
class VaultSigner(Signer):
    def sign(self, payload):
        # ... you get the idea

metadata = tuf.Metadata(...)
metadata.sign(VaultSigner(...))

[lukpueh]

@MVrachev, I thought you agreed with my arguments above for not implementing verification as method of the Signature class? (And even if we did, I wouldn’t store the public key on the Signature object but rather pass it to the verification method as argument.)