Quadratic processing time for array responses

Consider the following contract:

contract Test {
  uint32 constant SIZE = // size here
  uint256[SIZE] private array;
  function get() external view returns (uint256[SIZE] memory) {
    return array;
  }
}

When SIZE is > ~1000 elements, the time spent waiting for a response to a web3 call() to get() starts to grow in a super-linear fashion.

NumElements: Time

1000: 1.0s 2000: 2.9s 3000: 5.6s 4000: 9.2s 5000: 13.9s 6000: 19.4s 7000: 25.9s

I’ve tested this both with the truffle web3js client as well as the rust-web3 client, both against ganache-cli and truffle develop and I see the same behavior. ganache-cli vs truffle develop have nearly the same response times, which makes leads me to believe there is some super-linear processing in ganache-core.

The cpu is spiked to 100% on a single core during the processing time.

[domohawk]

I can confirm that geth does not seem to have this growth behavior, it seems to grow pretty linearly.

geth 1.10.6-stable, (both websockets & http)

NumElements	Time
1000	0.006s
5000	0.023s
25k	0.1s
50k	0.2s
100k	0.43s
500k	2.3s
1M	4.7s

[domohawk]

@davidmurdoch So using node v14 improves the times more, but still same growth behavior. Both websockets and http have the same times.

ganache@canary, node v14.17.5 (both websockets & http)

NumElements	Time
1	0.015s
1000	0.8s
5000	4.0s
25k	25s
50k	68s
100k	251s