Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[Optimization] Replay Buffers shouldn't use copy when using np.array

See original GitHub issue

https://github.com/DLR-RM/stable-baselines3/blob/23afedb254d06cae97064ca2aaba94b811d5c793/stable_baselines3/common/buffers.py#L346-L349

We call np.ndarray(x).copy(). This is unnecessary because np.array has the argument “copy” which is True by default.

https://numpy.org/doc/stable/reference/generated/numpy.array.html

import numpy as np
x = [1,2,3,4,5]
x1 = np.array(x)
x is x1
# False
x2 = np.array(x1)
x2 is x1
# False

Issue Analytics

State:
Created 3 years ago
Comments:10 (6 by maintainers)

Top GitHub Comments

1reaction

Miffylicommented, Jul 17, 2020

Ok, lets make another PR for optimizations like this 👍 . Lets wait until agent performance has been matched so things do not crisscross to badly.

0reactions

jarlvacommented, Aug 4, 2020

Pardon, it was a general suggestion, not related specifically to this issue.

Top Results From Across the Web

How does this Python memory optimization work?

ndarray data, and not actually storing each copy of the individual array within itself. import numpy as np a = np.ones((2,3)) b =...

Replay Buffers — rlpyt documentation - Read the Docs

Use of namedarraytuples makes it straightforward to write data of arbitrary ... If modifying a replay buffer, it might be easier to first...

4.5. Understanding the internals of NumPy to avoid ...

In this recipe, we will see how to avoid unnecessary array copies in order to save memory. In that respect, we will need...

Deep Q-Network (DQN)-II - Towards Data Science

This technique is called replay buffer or experience buffer. ... individual NumPy arrays with batch data in PyTorch tensors and copies them ...

Part I: On-policy learning and SARSA (3 points)

The policy we're gonna use is epsilon-greedy policy, where agent takes optimal action with probability $(1-\epsilon)$, otherwise samples action at random.