ray tune error with multiagent policy graph

My error happens in the run_experiments(), specifically

ray.tune.error.TuneError: ('Trials did not complete', [PPO_WaveAttenuationPOEnv-v0_0_lr=1e-05])
Closing connection to TraCI and stopping simulation.

My code is based on the mutliagent example “multiagent_stabilizing_the_ring” project. Bascially, I want to try multiple RL CAVs on the same ring road with a shared PPO policy. Please let me know if I understood something wrong. Different with that example, I set

env_name="WaveAttenuationPOEnv",
         scenario="LoopScenario", 

policy graph part was kept the same:

def gen_policy():
        return (PPOPolicyGraph, obs_space, act_space, {})

    # Setup PG with an ensemble of `num_policies` different policy graphs
    policy_graphs = {'av': gen_policy()}

    def policy_mapping_fn(_):
        return 'av'

    config.update({
        'multiagent': {
            'policy_graphs': policy_graphs,
            'policy_mapping_fn': tune.function(policy_mapping_fn),
            'policies_to_train': ['av']
        }
    })

Additionally, it’s still not clear for me that if the shared policy is defined on the single-agent states and actions or joint states and actions. Any help would be truly appreciated!!

[eugenevinitsky]

That is correct; batches are collected from both RL vehicles and used for the training.

[eugenevinitsky]

Ah, glad that helped. So, for each policy graph you can just construct your desired action space as is currently done in the action_space and observation_space methods in the MultiWaveAttenuationEnv. For example, if you want to control two accelerations at once, you might make the action space Box(low=min_accel, high=max_accel, shape=(2,)) which will tell the policy graph to have two values as the output of the neural network.