How can I use an optimizer with the extracted parameters?

I would like to use an optimizer with the extracted parameters but I keep getting some unexpected behavior like the parameters are still referring to the parameter list in the original model…

model = DropoutNet(in_dim, args.h_dim, out_dim).to(args.device)                                                                                                                                                                                                          optimizer = torch.optim.SGD(model.parameters(), lr=1, weight_decay=1e-4)      

params = OrderedDict(model.parameters())
p_opt = torch.optim.SGD(params.values(), lr=1e-3)

yhat = model(x, params=params)
loss = criterion(y, yhat)

for p in params.values():
  p.grad.add_(loss)

p_opt.step()
p_opt.zero_grad()                                                                     

If I do something like this and then print out the parameters, the parameters in the model will be the same as the ones in the params dict. Is there a better way to do this?

[egrefen]

@tristandeleu The post above inspired me to test higher with torchmeta dataloaders (allow users to use any torch optimizers or arbitrary third party modules instead of MetaModules). See this gist for an example: https://gist.github.com/egrefen/19ad0b65cf4d997a4b5bebf6e98b0562

It works beautifully 😄

[egrefen]

Hello @deltaskelta!

Thanks for the plug, @tristandeleu 😄 You are right that higher is your friend here, and AFAIK is totally compatible with Torchmeta, although it would primarily aim to supplant the use to MetaModule so you just use normal pytorch modules and patch them on the fly, but the data loaders would be valuable. I haven’t tried this yet, but it should work out of the box. And as of yesterday, you can pip install higher to install v0.2 👍

@deltaskelta please help me understand what you are doing a little more, and maybe I can advise on how to mix higher and TorchMeta here for fun and profit(?). It seems to me like you are computing a loss, and then adding it to the grads of your model parameters and then taking an optimizer step using those grads. My questions:

1. Is DropoutNet a “vanilla” torch.nn.Module instance here?
2. Why would p.grad not be None here, as no call to loss.backward is made, or to torch.autograd.grad, or equivalent? I’d expect the add_ to fail with an AttributeError.
3. Even if it doesn’t, why do you need the optimizer step to be differentiable here? It’s not clear what the gradients you’d be seeking to define after this step are.