Proposal: separate Predictor objects

Feature request

model.predict_f() currently recomputes everything (kernel matrix, Cholesky, triangular_solve) from scratch at each call. Instead of recomputing these quantities, we should provide some kind of predictor object that only does those computations that touch the new input points, and caches everything that can be precomputed.

Motivation

Why does this matter? We want GPflow to be as fast as it can be! Recomputing lots of cacheable quantities from scratch is inefficient and unnecessarily slow.

The slowness of predictions (particularly in a loop) is a well-known, recurring issue, e.g. https://github.com/GPflow/GPflow/issues/1554, https://github.com/GPflow/GPflow/issues/1189, https://github.com/GPflow/GPflow/issues/1030, https://github.com/GPflow/GPflow/issues/1003. We recently merged https://github.com/GPflow/GPflow/pull/1528 which was added so that a special-cased version of this proposal could be implemented down-stream. (Note that there are further efficiency gains possible that even #1528 does not allow for.) It would be much better to implement this properly, and make it available for all users out of the box.

Proposal

Describe the solution you would like Predictions should be efficient out-of-the-box for GPflow models. Details are open for discussion!

One option could be something like a model.predictor() method which returns an (optionally tf.function-decorated) closure that takes new input points Xnew and computes the prediction efficiently (having already precomputed and cached all that can be computed without access to the test inputs).

Questions:

• Should this be extended to the other predict functions (predict mean and var, predict log density)?
• How will we be able to cache all the quantities needed within the various conditional branches? (E.g. not just cholesky(Kuu), but also the solve against q_mu)

[joelberkeley-secondmind]

this ticket is introduced as: here’s a problem, implement this solution. Can I suggest: here’s a problem, solve the problem, here’s one possible way of doing it. So as to avoid biasing to particular solutions?

[st--]

@joelberkeley-secondmind we’re caching the quantities depending on the training data. There’s no caching of test points.