Add support for granular timeouts

🚀 Feature

The compiler environment uses timeouts on all operations. These timeouts are controlled by a ConnectionOpts class, which is default-constructed by the constructor. This feature request proposes adding the ability for user code to override these timeouts on a per-method basis by passing in an timeout kwarg to operations that can timeout.

Motivation

Timeout specification is currently very coarse-grained. E.g. ConnectionOpts. rpc_call_max_seconds specifies the timeout of all RPC operations, regardless of what the operation is, the size of the benchmark that is being processed, etc. A timeout that would be suitable for a fast operation like ending a session would be totally unsuitable for a slow operation like applying a long sequence of expensive actions on a large benchmark. Currently the only way to tweak these timeouts to match the different calls is to manually change the env.service.opts fields in between method calls, e.g.:

>>> env = gym.make("llvm-v0", connection_settings=ConnectionOpts(rpc_call_max_seconds=10))
>>> env.reset()
>>> env.connection.opts.rpc_call_max_seconds = 60
>>> env.step(0)
>>> env.connection.opts.rpc_call_max_seconds = 10
>>> env.reset()

Pitch

Add an optional timeout kwarg so that timeouts can be overriden on a per-method basis:

>>> env = gym.make("llvm-v0")
>>> env.reset(timeout=10)
>>> env.step(0, timeout=60)
>>> env.reset(timeout=10)

The following methods would receive this new argument:

• compiler_gym.envs.CompilerEnv.step
• compiler_gym.envs.CompilerEnv.multistep
• compiler_gym.envs.CompilerEnv.reset
• compiler_gym.envs.CompilerEnv.__init__

All subclasses / wrappers would need to handle this too.

Alternatives

We could also consider ditching the ConnectionOpts class entirely and pushing all of the configurable options out to the individual methods, but I see there are two drawbacks to this approach:

1. It makes it more difficult to support the use case where we don’t need granular timeout control, e.g. where the usere would just like to specify a longer timeout that can be applied to all operations.
2. Constructor timeouts would still need to saved so that they can used in fork() to propagate the same settings.

[ricardoprins]

Hi @ChrisCummins, I’d like to work on this.

[ChrisCummins]

Yep, that’s right!

Cheers, Chris