Initialization of num_data parameter in mll.VariationalELBO

In the Large-Scale Stochastic Variational GP Regression (CUDA) (w/ KISS-GP) notebook (https://gpytorch.readthedocs.io/en/latest/examples/05_Scalable_GP_Regression_Multidimensional/SVDKL_Regression_GridInterp_CUDA.html), shouldn’t the num_data parameter be initialized with batch_size rather than train_y.size(0)?

mll = gpytorch.mlls.VariationalELBO(likelihood, model.gp_layer, num_data=train_y.size(0), combine_terms=False)

[jacobrgardner]

@Akella17 If it helps, here’s an explanation of the normalization that is happening:

The ELBO for stochastic optimization is something like num_data / num_batch E_{q(f)}[log p(y | f)] - KL[q(u)||p(u)]. We divide the ELBO by the total number of data points, leading to 1 / num_batch E_{q(f)}[log p(y | f)] - 1 / num_data KL[q(u)||p(u)].

num_data should be the total number of data points you would pass over in an epoch, where-as num_batch is the number of data points in a single minibatch.

[gpleiss]

The short answer is no

The long answer: I’m not sure I totally understand the setup. From the GP-based reinforcement l’ve done, here’s the setup I used:

• Start out with no training data (num_data=0) For however long I want to train for:
  • Run a new trial/episode, where the model is conditioned on the data I’ve already collected. Gather new data
  • Create a new model/update the existing model with the new data (num_data = num_data + len_of_last_trial_epsiode). Train the model

Setting num_train to be the episode length (or approximate episode length) implies that your model is only being trained on a single episode, which is probably not what you want.