Confidence of classification output

Hello!

Thank you for your awesome work, I have been using gpytorch for a bit now and love it!

I was wondering how it is possible to obtain the confidence or variance of a classification output especially in the case of the DKL example, i.e. classification with multiple classes.

If there is no built-in way, the two ideas I had would be to either take the variance as output by the Gaussian variational distribution or to compute the variance of the categorical distribution, i.e. use p_i(1-p_i). Does gpytorch offer a standard way for deriving a confidence measure that I am missing here?

[jacobrgardner]

Variational inference (and therefore classification) follow the same pattern as exact regression, where passing data through the model in eval mode gets you p(f|D) (or in the variational setting q(f|D)), and passing that through the likelihood gets you p(y|D) (or q(y|D)).

In other words, something like:

model.eval()
preds_f = model(test_x)
preds_f.variance

Will get you variances of the latent function(s) f before they were mixed in the multiclass likelihood to form the categorical distribution. Note that if there are multiple GPs (for example in the standard SV-DKL implementation where an independent GP is used for each output feature of the neural network), this will get you variances for each GP, and you may need some means of combining the separate variances.

[cherepanovic]

what is about uncertainty? It is possible to get it for each classification?