Allow to opt out metadata swap for `isHistoric` queries

From my understanding, recently, the isHistoric flag is introduced to some RPC calls, causing a metadata swap that includes some overhead RPC calls (getRuntimeVersion and getHeader).

For the use case of substrate-api-sidecar, it would be very helpful if this metadata swap could be opted out of (i.e. appending a param isHistoric=false). So far we are seeing a significant performance regression, even with the optimization introduced here: https://github.com/polkadot-js/api/commit/637069d723bf9a8afcd42af8b3162a20737ff501 - even though I would have thought it would have fixed performance issues because the endpoint services make every call possible at the same block hash.

Currently Sidecar has its own method to swap out metadata and we are working on a way to handle edge cases like runtime upgrade blocks while only having a minimal performance penalty (see here: https://github.com/paritytech/substrate-api-sidecar/pull/232). We are trying to avoid the overhead of call to getHeader, which the new built in swap implicitly does with every query to a different block hash. This is especially important because the longer an endpoint takes to respond, the higher the memory footprint of Sidecar grows with concurrent requests, and just all around lower throughput/ higher latency - which is becoming a concern as the users of Sidecar experience quickly scaling traffic.

Here are the response times from using our own metadata swapping with v1.30.1 and querying the same block endpoint 5 times in a row. The first is quite long because we have to fetch the metadata since we don’t have any default metadaRpc (branch master, commit 3dfe6dc353da4027cae8ee63cc0bf5244a6490ca) :

And here are the response times when I disable our own metadata swap and use v1.31.beta.x (branch: zeke-tst-polk-js commit: bb52ae971a51701b4cdbc7c3f773ce57b796d8a3)

Also for reference in case you want to see exact calls for this endpoint:

• controller: https://github.com/paritytech/substrate-api-sidecar/blob/zeke-tst-polk-js/src/controllers/v0/v0blocks/BlocksController.ts#L90
• service: https://github.com/paritytech/substrate-api-sidecar/blob/zeke-tst-polk-js/src/services/v0/v0blocks/BlocksService.ts#L39
• metadata swapping: https://github.com/paritytech/substrate-api-sidecar/blob/master/src/services/AbstractService.ts#L16

Also let me know if there is anything related to this that I can help implement on the polkadot-js side. I am not that familiar with the RPC architecture but happy to help and learn more about it.

[jacogr]

Closing an “wontfix”, sadly the native support is not going away since it has been the #1 support burden in the last 9 months. Additionally there are a couple of issues waiting that needs this (e.g. consts.<module>.<method>.at(blockHash) and keys.at(...)/entries.at(...)

Fo specific suggestions on additional improvements regarding performance (here and elsewhere), all ears. The last suggestion here actually helped a lot.

[polkadot-js-bot]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue if you think you have a related problem or query.