Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[RLlib] TrajectoryView positive shifts doesn't work

See original GitHub issue

Hello, I am trying to access the next action during training. I updated the requirement with

"next_actions": ViewRequirement( SampleBatch.ACTIONS, shift=1, used_for_training=True)

However it doesn’t work because, when building the batch, data gets an empty list (see line 141)

https://github.com/ray-project/ray/blob/ad639f12d85297690b1190837e55ee47ce554e60/rllib/evaluation/collectors/simple_list_collector.py#L127-L144

I think that line 141 should be changed to

data = np_data[data_col][self.shift_before + shift:] + padding

@sven1977 could you take a look ? Thanks !

Issue Analytics

State:
Created 3 years ago
Comments:5 (5 by maintainers)

Top GitHub Comments

1reaction

sven1977commented, Nov 25, 2020

Hey @raphaelavalos , thanks for filing this issue! Great stuff that users are already trying to do cool (and completely unexpected 😄 ) things with the API. Taking a look right now. …

0reactions

raphaelavaloscommented, Nov 30, 2020

Thank you @sven1977 !

Top Results From Across the Web

Sample Collections and Trajectory Views — Ray 2.2.0

The trajectory view API lets models define these requirements and lets RLlib gather the required data for the forward pass in an efficient...

Potential bug in trajectory view API for multiagent envs - Ray

Does the trajectory view API support multiagent environments? I'm currently hitting the following internal error from trainer.train():

Examples — Ray 2.2.0

This blog describes RLlib's new “trajectory view API” and how it enables implementations of GTrXL (attention net) architectures.

How To Customize Policies — Ray 2.2.0

To simplify the definition of policies, RLlib includes Tensorflow and ... The above basic policy, when run, will produce batches of observations with...

RLlib: Industry-Grade Reinforcement Learning — Ray 2.2.0

RLlib is an open-source library for reinforcement learning (RL), offering support for ... RLlib does not automatically install a deep-learning framework, ...