bookmarks in query streams, with offset argument to websocket endpoints

Starting at the end of non-live data (#4461) we will include a "offset" as discussed in #3878. This will incompatibly change the liveness marker from {"live": true} to the first occurrence of an offset: {"events": [...], "offset": ...}, where [...] may be empty.

As implied by the above, not every response block will include an offset. We only promise to provide offsets once we reach the end of non-live data. In fact, we promise not to provide offsets until we reach the end of non-live data.

The current query union format is {templateIds, query} | [{templateIds, query}...], and fetch by key format is [{templateId, key}]; @hurryabit to determine where the nicest place to put offset is in these datatypes, noting that we can only reasonably have one offset for all queries in the union. The behavior of specifying a non-existent offset will be passed through from ledger API.

The only intended effect of this change is to make streams reconnect-able; the remaining stream semantics will not change.

This will disable “phantom archive” filtering entirely for resumed contract key streams, while it will remain in place for fetch streams for which offset is not supplied. Refining this behavior should be addressed in a separate issue, as @gerolf-da has noted various complicating factors.

Depends on #4465.

cc @dtanabe @lima-da

• how to pass offset, especially in case of [union...]?
• how to represent LedgerBegin? null

[leo-da]

@S11001001 how about

{events: [...], live: false, offset=<last event offset>}
{events: [...], live: true,  offset=<last event offset>}
{events: [...], live: true,  offset=<last event offset>}

[lima-da]

ok, then I am fine with it if we can strictly define that only -> {offset: 1} -> [query1, query2,…] <- {“events”: …} or -> [query1, query2,…] <- {“events”: …} would work.