Error in Deep_Kernel_Learning_DenseNet_CIFAR_Tutorial.ipynb

In the tutorial example for Deep Kernel Learning on the CIFAR datasets there is an issue in the test() function. The following error is raised:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-8-30679d80b21c> in <module>
      3     with gpytorch.settings.use_toeplitz(False), gpytorch.settings.max_preconditioner_size(0):
      4         train(epoch)
----> 5         test()
      6     state_dict = model.state_dict()
      7     likelihood_state_dict = likelihood.state_dict()

<ipython-input-7-91f72c08681a> in test()
     35         with torch.no_grad():
     36             output = likelihood(model(data))
---> 37             pred = output.argmax()
     38             correct += pred.eq(target.view_as(pred)).cpu().sum()
     39     test_loss /= len(test_loader.dataset)

AttributeError: 'Categorical' object has no attribute 'argmax'

It appears that the output of likelihood() should be an instance of torch.Tensor, and not torch.distributions.categorical.Categorical. This is consistent with gpytorch.likelihoods.SoftmaxLikelihood. Should .argmax() be called on one of the attributes of output, or is there another recommended best practice to return the predictions?

[messiest]

@jacobrgardner I had been trying pred = output.probs.argmax(), but the dimension was wrong and would raise the following RuntimeError:

RuntimeError: shape '[]' is invalid for input of size 64

@gpleiss Excellent, works like a charm now. (Fixed locally).

[gpleiss]

@messiest - just updated the example notebook. You might want to check out the new optimizer settings. In my latest experiments I’ve noticed that you want a much smaller learning rate on the kernel hyperparameters, but the GP’s variational parameters should have a pretty high learning rate.