Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
Question about second-order gradients for GRU
See original GitHub issueHi, I was trying to use the package for obtaining second-order gradients through the optimization process of a model with GRU units each followed by a linear layer. However, when I check torch.autograd.grad(loss, learner_fmodel.parameters(time=0)) I get the error RuntimeError: One of the differentiated Tensors appears to not have been used in the graph. Set allow_unused=True if this is the desired behavior.. With allow_unused=True, I see that the gradients with respect to the GRU parameters are None whereas the gradient with respect to the linear layer has values. I was wondering if this is indeed supported for GRU?
Issue Analytics
- State:
- Created 4 years ago
- Comments:14 (9 by maintainers)
Top Results From Across the Web
Top Related Medium Post
Top Related StackOverflow Question
Troubleshoot Live Code
Top Related Reddit Thread
Top Related Hackernoon Post
Top Related Tweet
Top Related Dev.to Post
Top Related Hashnode Post
Just to update you, I can also reproduce this error. Eyeballing the current pytorch rnn code, I’m not sure why this is happening, so will need to loop in someone from the pytorch team.
I note that the unit tests for RNNs in higher is incomplete and wouldn’t catch this, so I will fix that first, and then try to get this progressed.
This is a blocking issue for my own research so will try to get this sorted ASAP.
@egrefen It works for me as well. Thanks for the effort!