[Docs] Usage of gpytorch.settings.use_toeplitz in SKIP GPs

📚 Documentation/Examples

Example in question: https://docs.gpytorch.ai/en/v1.1.1/examples/02_Scalable_Exact_GPs/Scalable_Kernel_Interpolation_for_Products_CUDA.html

In code cell 7, train is called with

...
with gpytorch.settings.use_toeplitz(True):
    %time train()
...

Later, however, inside train this same context is set to False.

...
with gpytorch.settings.use_toeplitz(False), gpytorch.settings.max_root_decomposition_size(30):
    # Get output from model
    output = model(train_x)
...

Two questions:

• The call to optimizer.step() remains out of this context. Is this intentional? Does it matter?
• Doesn’t setting this to False again undo the previous context? As I understand, the kernel we are using for grid interpolation should be utilizing Toeplitz structure?

Further, I also have this warning which seems to concern with usage of nonzero() in PyTorch.

lib/python3.7/site-packages/gpytorch/utils/interpolation.py:119: UserWarning: This overload of nonzero is deprecated:
        nonzero()
Consider using one of the following signatures instead:
        nonzero(*, bool as_tuple) (Triggered internally at  /pytorch/torch/csrc/utils/python_arg_parser.cpp:766.)
  left_boundary_pts = (lower_grid_pt_idxs < 0).nonzero()

Thanks!

Versions:

GPyTorch: 1.1.1 PyTorch: 1.6.0

[KeAWang]

No optimizer.step() will just update the parameters by the gradients so it isn’t affected by the context

[activatedgeek]

No, I meant whether the optimizer call is within context or outside, doesn’t matter. Is this correct?