Support divulged contracts in a distributed ledger

Divulged contracts in a distributed ledger (i.e. multi-participant topology) are currently not supported. This is witnessed by a foreign key constraint error in the Ledger API Server, when it tries to insert entries into the contract_divulgences table. After a discussion with @oliverse-da and @meiersi-da, we propose the following steps to support divulged contracts:

1. Split the current contracts table into contracts_data and contracts.
   1. contracts_data only holds the relation (ContractId, ContractInstance)
   2. contracts only holds the metadata (i.e. what it currently holds minus the contract instance data)
2. The type of TransactionAccepted#referenceContracts is changed to List[(AbsoluteContractId, VersionValue[AbsoluteContractInst)]
3. PersistenceEntry.Transaction is extended with the list of referenced contracts from TransactionAccepted.
4. The class c.d.p.s.s.l.s.d.Contract needs to change the fields let, transactionId to be of type Option.
5. Stop using ActiveContract internally in ActiveContractsManager, but only use Contract.
6. Rename LedgerDao#lookupActiveContract to LedgerDao#lookupPotentiallyActiveContract.
7. Only perform the LET contract check if the LET is actually available. For divulged contracts, we can only assume that it is active, as we don’t have any further information.
8. When adding contracts coming from a TransactionAccepted state update, a “divulged” contract can be upgraded to a disclosed contract by:
   1. checking that the existing ContractInst value in contracts_data is exactly the same as the to be inserted value from the transaction
   2. additional metadata to the already known entry in contracts_data is written to the contracts table.
9. A new migration script for postgres must be added to split the contracts table into two tables.

Points to think about:

• If a contract is disclosed AFTER it has been divulged (i.e. more metadata like LET and ledger offset is now available), does this have an effect on consistency assumptions? For example, it is possible that Tx(N).LET < Tx(N-1).LET, where Tx(N) is Transaction at offset N.

[meiersi-da]

Thanks @gerolf-da .

Potential typo in Point 2: should be List[NodeCreate[AbsoluteContractId, VersionValue[AbsoluteContractInst]]]

On point to think about:

• this is a ledger consistency violation that can have been detected by the signatories of the divulged contract. We expect ledger implementations to avoid such consistency violations as part of the process of committing a transaction.

[oliverse-da]

Fwiw, at the time we implemented multi-domain divulgence, we could not identify a scenario for item 8 (for a previously divulged contract to become fully known to a participant). Multi-domain scenarios will at some point have such a scenario as contracts are transferred in an out of domains, but we will follow up on that separately (#3334) once multi-domain requirements are sufficiently understood and formalized. (Just thought I’d point this out here as fyi.)

cc @andreaslochbihler-da