[rllib] DDPG opensim env issue

System information

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): ubuntu 16.04
• Ray installed from (source or binary): source
• Ray version: 0.5.0
• Python version: 3.6.1
• Exact command to reproduce:

Describe the problem

Executing DDPG in Pendulum-v0 environment has no problem.

However, when i execute DDPG in Prosthetics environment, error occurs during gradient calculation. Do I have to add some additional code in custom environment to solve this problem?

Source code / logs

import ray 
import ray.tune.registry import register_env
from ray.rllib.agents import ddpg
from osim.env import ProstheticsEnv

def env_creator(env_config):
    return ProstheticsEnv(False)

register_env("prosthetics", env_creator)
ray.init("<head-ip:port>")

agent = ddpg.DDPGAgent(env="prosthetics", config={"num_workers": 140, "gpu": True})

for i in range(1000):
    result = agent.train()
    print(result)
    if i % 10 == 0:
        agent.save()

[ericl]

This is odd, you shouldn’t be getting MultiAgentBatch types there. Could you print the types (or even better the full contents) of each s on this line?: https://github.com/ray-project/ray/blob/d01dc9e22d5e8625ae6ac49e2e689eebf472b5f8/python/ray/rllib/evaluation/sample_batch.py#L224

[whikwon]

I’ve ran local desktop and it runs. I think one of my VM has environment problem something like that. I’ll find out and let you know if it belongs to ray. Please close this issue.