Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
Alternative version / replacement of Replay Plugin
See original GitHub issueCurrently, Replay plugin keeps an external memory of patterns which are concatenated to the current step dataset. Minibatches are then sampled from this new dataset.
We could provide an additional Replay version or substitute the current one by concatenating to each minibatch a random subset of the external memory. In this way, there is no need to concatenate to the entire dataset and replay is more effective (even if it costs more in terms of computation).
This can be done in before_forward which is only applied to the training phase.
Issue Analytics
- State:
- Created 3 years ago
- Reactions:3
- Comments:15 (3 by maintainers)
Top Results From Across the Web
Top Related Medium Post
Top Related StackOverflow Question
Troubleshoot Live Code
Top Related Reddit Thread
Top Related Hackernoon Post
Top Related Tweet
Top Related Dev.to Post
Top Related Hashnode Post
Just a quick note: you can already split the experiences in datasets of 1 example each, so technically you could have a stream of data without “boundaries”. Of course, that would be very inefficient in pytorch. I agree with the idea of fixing this strategies side!
I see, but I don’t think we would have to remove experiences. The data stream can still be divided in experiences, it’s just the methods should not take advantage or even be aware of the boundaries of experiences. (e.g. the real-life data stream can happen to be just like the typical Split-MNIST scenario) So I think a simple solution with a custom dataloader like @AntonioCarta proposed, could already solve this issue (: