Parse header with proper hash (tendermint-rpc + proto-ts)

Proto-ts defines Header.time (from tendermint/types) as Date, which has millisecond precision. The rpc returns nanosecond precision, which we can capture in tendermint-rpc (this is for the Header object). But we will loose it when constructing the ts-proto object. This must be identical for the header hash to match the signatures. (At least I think so, I need to ensure they do hash the nanosecond result).

This thought came up when I hit some failing ibc tests with: “header failed basic validation: commit signs block 9E8DAB852CE462075B06FE258B0B84E66E480F54644E30256BC76EFE0C44E119, header is block 6C3DE71B5E0376E364B7CEBD22D0917F4B0EC391A3A1430CD847DB59EF52E4D1”

I am also a bit confused by some queries:

curl http://localhost:26658/commit?height=3725

{
  "jsonrpc": "2.0",
  "id": -1,
  "result": {
    "signed_header": {
      "header": {
        "version": {
          "block": "11"
        },
        "chain_id": "simd-testing",
        "height": "3725",
        "time": "2021-02-10T22:47:15.968851173Z",
        "last_block_id": {
          "hash": "949532143488917A7D920A92A599CD3C90AD5979AFCE07AE790EE36AEEF0CCCA",
          "parts": {
            "total": 1,
            "hash": "D79BB4BED310C0A67656B8B30D1FEA35FC105D23660098BBF0F4608D5E9AB753"
          }
        },
        "last_commit_hash": "F9E6B97A12EF491B46B039F827A8CA0FB121B1959DEA76357FD29659AD58C91B",
        "data_hash": "E3B0C44298FC1C149AFBF4C8996FB92427AE41E4649B934CA495991B7852B855",
        "validators_hash": "C73C1C6A829C9BF3F574CD00FC1D9EB54FC2E378FAC4D0A2F57878CDF74DFEF7",
        "next_validators_hash": "C73C1C6A829C9BF3F574CD00FC1D9EB54FC2E378FAC4D0A2F57878CDF74DFEF7",
        "consensus_hash": "048091BC7DDC283F77BFBF91D73C44DA58C3DF8A9CBC867405D8B7F3DAADA22F",
        "app_hash": "297B39D7C25115AB92A28D70FBD3608D59167EDD88AE1126A4C0D0EEA14D7DB2",
        "last_results_hash": "E3B0C44298FC1C149AFBF4C8996FB92427AE41E4649B934CA495991B7852B855",
        "evidence_hash": "E3B0C44298FC1C149AFBF4C8996FB92427AE41E4649B934CA495991B7852B855",
        "proposer_address": "E359FA31E749FFDC571AA4E5F629F70CD53D328A"
      },
      "commit": {
        "height": "3725",
        "round": 0,
        "block_id": {
          "hash": "9E8DAB852CE462075B06FE258B0B84E66E480F54644E30256BC76EFE0C44E119",
          "parts": {
            "total": 1,
            "hash": "0C3528C46715A4775C44DF1C76184145EE0A2581794FF08F74B90DBD8FD2EA2E"
          }
        },
        "signatures": [
          {
            "block_id_flag": 2,
            "validator_address": "E359FA31E749FFDC571AA4E5F629F70CD53D328A",
            "timestamp": "2021-02-10T22:47:17.015519194Z",
            "signature": "GEn7xg/fxcVYNVBMVYlzu0ZGMXEUrt8JtjIJic7vdqIBeDF3FiWpgF52/FLhEnGL24AviNptuM15LJxdnq/8CQ=="
          }
        ]
      }
    },
    "canonical": true
  }
}

Well, the header doesn’t ever show it’s own block_id, just the last one (building the chain). Let’s see what the next header claims that 3725 was:

curl http://localhost:26658/commit?height=3726 | jq .result.signed_header.header.last_block_id

{
  "hash": "9E8DAB852CE462075B06FE258B0B84E66E480F54644E30256BC76EFE0C44E119",
  "parts": {
    "total": 1,
    "hash": "0C3528C46715A4775C44DF1C76184145EE0A2581794FF08F74B90DBD8FD2EA2E"
  }
}

Yes, it agrees. And what was the header object we got (parsed):

{
  chainId: 'simd-testing',
  height: Long { low: 3725, high: 0, unsigned: false },
  version: {
    block: Long { low: 11, high: 0, unsigned: false },
    app: Long { low: 0, high: 0, unsigned: true }
  },
  time: 2021-02-10T22:47:15.968Z,
  lastBlockId: {
    hash: Uint8Array(32) [
      148, 149,  50,  20,  52, 136, 145,
      122, 125, 146,  10, 146, 165, 153,
      205,  60, 144, 173,  89, 121, 175,
      206,   7, 174, 121,  14, 227, 106,
      238, 240, 204, 202
    ],
    partSetHeader: undefined
  },
  lastCommitHash: Uint8Array(32) [
    249, 230, 185, 122,  18, 239,  73, 27,
     70, 176,  57, 248,  39, 168, 202, 15,
    177,  33, 177, 149, 157, 234, 118, 53,
    127, 210, 150,  89, 173,  88, 201, 27
  ],
  dataHash: Uint8Array(32) [
    227, 176, 196,  66, 152, 252,  28,
     20, 154, 251, 244, 200, 153, 111,
    185,  36,  39, 174,  65, 228, 100,
    155, 147,  76, 164, 149, 153,  27,
    120,  82, 184,  85
  ],
  validatorsHash: Uint8Array(32) [
    199,  60,  28, 106, 130, 156, 155,
    243, 245, 116, 205,   0, 252,  29,
    158, 181,  79, 194, 227, 120, 250,
    196, 208, 162, 245, 120, 120, 205,
    247,  77, 254, 247
  ],
  nextValidatorsHash: Uint8Array(32) [
    199,  60,  28, 106, 130, 156, 155,
    243, 245, 116, 205,   0, 252,  29,
    158, 181,  79, 194, 227, 120, 250,
    196, 208, 162, 245, 120, 120, 205,
    247,  77, 254, 247
  ],
  consensusHash: Uint8Array(32) [
      4, 128, 145, 188, 125, 220,  40,
     63, 119, 191, 191, 145, 215,  60,
     68, 218,  88, 195, 223, 138, 156,
    188, 134, 116,   5, 216, 183, 243,
    218, 173, 162,  47
  ],
  appHash: Uint8Array(32) [
     41, 123,  57, 215, 194,  81,  21,
    171, 146, 162, 141, 112, 251, 211,
     96, 141,  89,  22, 126, 221, 136,
    174,  17,  38, 164, 192, 208, 238,
    161,  77, 125, 178
  ],
  lastResultsHash: Uint8Array(32) [
    227, 176, 196,  66, 152, 252,  28,
     20, 154, 251, 244, 200, 153, 111,
    185,  36,  39, 174,  65, 228, 100,
    155, 147,  76, 164, 149, 153,  27,
    120,  82, 184,  85
  ],
  evidenceHash: Uint8Array(32) [
    227, 176, 196,  66, 152, 252,  28,
     20, 154, 251, 244, 200, 153, 111,
    185,  36,  39, 174,  65, 228, 100,
    155, 147,  76, 164, 149, 153,  27,
    120,  82, 184,  85
  ],
  proposerAddress: Uint8Array(20) [
    227,  89, 250, 49, 231,  73,
    255, 220,  87, 26, 164, 229,
    246,  41, 247, 12, 213,  61,
     50, 138
  ]
}

The header hash discrepancy seems to either be due to missing nanoseconds on the time, or missing partSetHeader on blockId (or both?)

[ethanfrey]

I found this in ts-proto:

With --ts_proto_opt=useDate=false, fields of type google.protobuf.Timestamp will not be mapped to type Date in the generated types

I guess I need this to compile the tendermint types. And a DateWithNanoseconds type that converts to/from this might be useful as well.

I still need to check about the missing info in blockId

[webmaster128]

With --ts_proto_opt=useDate=false, fields of type google.protobuf.Timestamp will not be mapped to type Date in the generated types

This sounds like a reasonable default in our context. With this set, timestamps become a nested structure like this: https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/timestamp.proto#L136. Would be good to have a DateWithNanoseconds constructor that can easily convert from this.