Custom Policy Network ---Observation dimensions

❓ Question

` def forward(self, observations: th.Tensor) -> th.Tensor:

    print(observations.size())

    return self.linear(self.cnn(observations))`

When the program is running, the output is ‘torch.Size([1, , ])’'for one period of time, and the output is ‘torch.Size([128, , ])’ for another period of time, and the loop continues. （batch_size is 128）Why is this so？Is it possible to make the observation dimension constant?

Checklist

• I have checked that there is no similar issue in the repo
• I have read the documentation
• If code there is, it is minimal and working
• If code there is, it is formatted using the markdown code blocks for both code and stack traces.

[qgallouedec]

So that’s what I thought. It’s not the size of the observation that changes, but the size of the input batch. It is 64 during training (batch_size=64), and 1 during the interaction phase.

[qgallouedec]

When the program is running,

• If code there is, it is minimal and working

What program? Please provide a minimal and working code.

I think you are confusing the batch size and the observation size. The input tensor in the network have a size batch_size x obs_size. During the evaluation, only one observation is used as input of the network. Consequently, first dimension is 1.