[Bug] Unable to estimate independent scale parameters for each input

I have a spatio-temporal GP with three input dimensions (2 spatial coordinates plus time) and am trying to use ARD to have separate lengthscale parameters for each, as well as separate scale kernels for the spatial vs the temporal inputs. My covariance function looks like this:

self.covar_module = (
            gpytorch.kernels.ScaleKernel(
                gpytorch.kernels.MaternKernel(nu=1/2, 
                                          active_dims=torch.tensor([0, 1]), 
                                          ard_num_dims=2,
                                          batch_size=batch_size),
                active_dims=torch.tensor([0, 1]), 
                ard_num_dims=2,
                batch_size=batch_size)
            * gpytorch.kernels.ScaleKernel(
                gpytorch.kernels.RBFKernel(active_dims=torch.tensor([2]),
                                          batch_size=batch_size),
                active_dims=torch.tensor([2]),
                batch_size=batch_size))

However, when I fit the model, looking at the estimated parameters afterwards shows that the same scale is being estimated for both components:

likelihood.noise_covar.raw_noise tensor([-5.4645], device='cuda:0')
covar_module.kernels.0.raw_outputscale tensor(-3.7285, device='cuda:0')
covar_module.kernels.0.base_kernel.raw_lengthscale tensor([[2.4628, 3.4885]], device='cuda:0')
covar_module.kernels.1.raw_outputscale tensor(-3.7285, device='cuda:0')
covar_module.kernels.1.base_kernel.raw_lengthscale tensor([[3.7490]], device='cuda:0')

as having the same value to 4 decimal places seems unlikely otherwise.

Is this user error on my part, or is this a bug? Is it possible to estimate different scales for each input feature?

[fonnesbeck]

There is a somewhat confusing API issue here, however. Namely, its not clear why a kernel should be wrapped at all by ScaleKernel when it is simply pulled outside as a multiplier. In other words,

ScaleKernel(MaternKernel()) * RBFKernel()

seems superfluous when all that ends up happening is:

ScaleKernel() * MaternKernel() * RBFKernel()

Is there anything beyond this going on to the passed kernel that does not end up also being applied to the second kernel?

[fonnesbeck]

Got it, thanks.