Weights of monkeypatched module get reset by a forward pass after being changed manually

Any computed update to the weights of a monkeypatched module (not with a differentiable optimizer) doesn’t behave as expected:

data = torch.randn(32,10)  
module = torch.nn.Linear(10,10)  
fmodel = higher.monkeypatch(module,copy_initial_weights=True)  
fmodel.UpdateWeights() # some update to the weights, for a simple example: fmodel.weight = fmodel.weight * 2.
fmodel.forward(data) # weights are now reset as if fmodel.UpdateWeights() never happened

Edit: I’ve noticed that fmodel.named_parameters() and fmodel._parameters contain the correct updated parameters whereas fmodel.parameters() and fmodel.fast_params contain the old parameter values

[egrefen]

Regarding a workaround, you can do fmodel.parameters() to get the current state of the “fast” parameters. If you want to apply a (non in-place) function to a subset of them, do so on the returned iterable, forming a new iterable new_params with the same number of parameter tensors (i.e. including the unmodified ones), and then update the parameters by calling fmodel.update_params(new_params).

This isn’t really a pattern I would encourage or will guarantee to support, but it should work for now.

[egrefen]

Addressed in #43. Please re-open issue if it’s not working for your use case.