Spike: Investigate how to handle versioned datastructures [3D]

Description

As part of #3356, we’ll need to figure out how to handle data structures (like BeaconState) that change format across forks.

See the eth2 spec repo for details on the proposed changes to BeaconState. Basically, 2 fields on BeaconState (previous_epoch_attestations and current_epoch_attestations) will be replaced with 2 new fields: previous_epoch_participation and current_epoch_participation. Additional data structure changes are involved in implementing light-client support as well.

Issues we want to consider:

• How are versioned data structure classes organized?
  • Do we have completely distinct versions: BeaconStateV1, BeaconStateV2?
  • Do we try to somehow generify the optional fields: BeaconState<TVersionedFields>?
  • Do we have an overarching BeaconState interface with more specific implementations: BeaconStateV1 implements BeaconState?
• How does versioned core logic that operates on a BeaconState work?
• How is (de)serialization handled by the Database
  • Likely we will need to delegate to a deserializer that we pull from the SpecProvider. This deserializer could take the serialized Bytes, extract the state’s slot or fork info, use that to find the right deserializer instance, and then deserialize the state to the right type.

Outcomes:

• Propose a general plan for how to address the outlined issues, get feedback from the team
• If appropriate, think about tickets / subtasks we might want to add

Time box: 3 days.

[mbaxter]

Another option could be something like:

public interface BeaconState {
  // Common fields
  UInt64 slot();
  ListViewRead<Validator> validators();
  ListViewRead<UInt64View> getBalances();
  // etc

  // Variable fields
  Optional<SSZList<PendingAttestation>> maybeGetPrevious_epoch_attestations();
  Optional<SSZList<PendingAttestation>> maybeGetCurrent_epoch_attestations();

  Optional<SSZList<SSZVector<BitView>>> maybeGetPrevious_epoch_participation();
  Optional<SSZList<SSZVector<BitView>>> maybeGetCurrent_epoch_participation();
}

public interface BeaconStateGenesis extends BeaconState {
  SSZList<PendingAttestation> getPrevious_epoch_attestations();
  SSZList<PendingAttestation> getCurrent_epoch_attestations();
}

public interface BeaconStateHF1 extends BeaconState {
  SSZList<SSZVector<BitView>> getPrevious_epoch_participation();
  SSZList<SSZVector<BitView>> getCurrent_epoch_participation();
}

[Nashatyrev]

Likely we will need to delegate to a deserializer that we pull from the SpecProvider. This deserializer could take the serialized Bytes, extract the state’s slot or fork info, use that to find the right deserializer instance, and then deserialize the state to the right type.

We may potentially have structures modified in the future which doesn’t contain slot info. So probably makes sense to come up with some generic solution? E.g. prepend db record with a single version byte