Use batching/chunking timers

There are multiple places within Autobahn (at the WebSocket and at the WAMP level) where we use timers sticked to protocol instances.

When running massive numbers of connections, this can lead to significant CPU load.

Since these timers don’t need to be precise, and hence we can conserve CPU by batching timeouts into buckets (and process timeout callbacks within a bucket in chunks).

The underlying functionality will be in txaio: https://github.com/crossbario/txaio/issues/66

Once that is done, we should use them here.

[meejah]

With the changes from https://github.com/crossbario/txaio/pull/67 I hacked together some quick numbers after replacing all the call_laters in websocket stuff to batched timeouts.

This is just a local test on an 8-core machine, ramping up to 20k do-nothing websocket connections (in one client process) with 10s auto-pings and 60s ping-timeouts enabled and then idling for a while. I’m showing the CPU usage of the server process as grok’d by collectl every second (there’s almost no green impulses in the latter part of the graph because the CPU usage is very low).

Conclusion: well worth the batching optimization. I will do a few more tests tomorrow playing with some of the batching parameters (this was 5s bucket size, chunked at 1000 per yield). The high CPU usage on the left is during the ramp-up phase…

[oberstet]

@meejah alright, all of this is released in https://pypi.python.org/pypi/autobahn/0.14.0