Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

singleagent_merge training fails

See original GitHub issue

Bug Description

When I train singleagent_merge, it fails

Bug Reproduce

python train.py singleagent_merge

Output Log

Traceback (most recent call last):
  File "/home/intel/anaconda3/envs/flow/lib/python3.6/site-packages/ray/tune/trial_runner.py", line 498, in _process_trial
    result = self.trial_executor.fetch_result(trial)
  File "/home/intel/anaconda3/envs/flow/lib/python3.6/site-packages/ray/tune/ray_trial_executor.py", line 342, in fetch_result
    result = ray.get(trial_future[0])
  File "/home/intel/anaconda3/envs/flow/lib/python3.6/site-packages/ray/worker.py", line 2247, in get
    raise value
ray.exceptions.RayTaskError: [36mray_PPO:train()[39m (pid=14406, host=intel-pc)
  File "/home/intel/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/agents/trainer.py", line 369, in train
    raise e
  File "/home/intel/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/agents/trainer.py", line 358, in train
    result = Trainable.train(self)
  File "/home/intel/anaconda3/envs/flow/lib/python3.6/site-packages/ray/tune/trainable.py", line 171, in train
    result = self._train()
  File "/home/intel/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/agents/trainer_template.py", line 126, in _train
    fetches = self.optimizer.step()
  File "/home/intel/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/optimizers/multi_gpu_optimizer.py", line 140, in step
    self.num_envs_per_worker, self.train_batch_size)
  File "/home/intel/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/optimizers/rollout.py", line 29, in collect_samples
    next_sample = ray_get_and_free(fut_sample)
  File "/home/intel/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/utils/memory.py", line 33, in ray_get_and_free
    result = ray.get(object_ids)
ray.exceptions.RayTaskError: [36mray_RolloutWorker:sample()[39m (pid=14402, host=intel-pc)
  File "/home/intel/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/evaluation/rollout_worker.py", line 453, in sample
    batches = [self.input_reader.next()]
  File "/home/intel/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/evaluation/sampler.py", line 56, in next
    batches = [self.get_data()]
  File "/home/intel/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/evaluation/sampler.py", line 97, in get_data
    item = next(self.rollout_provider)
  File "/home/intel/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/evaluation/sampler.py", line 313, in _env_runner
    soft_horizon)
  File "/home/intel/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/evaluation/sampler.py", line 401, in _process_observations
    policy_id).transform(raw_obs)
  File "/home/intel/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/models/preprocessors.py", line 166, in transform
    self.check_shape(observation)
  File "/home/intel/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/models/preprocessors.py", line 61, in check_shape
    self._obs_space, observation)
ValueError: ('Observation outside expected value range', Box(280,), array([ 0.00000000e+00,  4.48674389e-03,  4.69197036e-02,  3.16010523e-05,
        0.00000000e+00,  0.00000000e+00,  4.69065457e-02,  3.10963661e-02,
        1.88532636e-01,  4.23515771e-01,  3.42134161e-02,  7.35134037e-02,
        7.60705448e-06,  6.90067447e-03,  3.24158843e-01,  3.17126994e-01,
        3.64110069e-01,  4.02902781e-01,  0.00000000e+00,  1.76898596e-03,
        8.69775084e-03,  7.68905950e-03,  0.00000000e+00,  5.08732856e-04,
        8.40520125e-02,  3.60284041e-02,  0.00000000e+00,  3.99014638e-01,
        1.32984023e-01,  0.00000000e+00,  2.96668282e-02,  1.89124059e-03,
        0.00000000e+00,  0.00000000e+00,  1.41107882e-05,  6.18246402e-05,
        0.00000000e+00,  0.00000000e+00,  0.00000000e+00,  0.00000000e+00,
        0.00000000e+00,  0.00000000e+00,  3.24483209e-01,  4.01030390e-01,
        4.71162745e-02,  4.70089797e-04,  0.00000000e+00,  2.85571169e-02,
        2.87845099e-03,  2.83716878e-02,  2.21626102e-02,  5.26387533e-02,
        0.00000000e+00,  0.00000000e+00,  2.28072147e-02,  5.68610825e-02,
        3.24224414e-02,  6.54499515e-01,  9.06251461e-03,  4.06600015e-02,
        6.74082556e-01,  6.99267902e-01,  5.60383461e-01,  6.35572339e-01,
        1.13066070e-01,  2.47346859e-01,  2.77777281e-03,  2.79191731e-02,
        3.33666667e-03,  3.07541214e-02,  3.33666667e-03,  2.83906531e-02,
        1.30654777e-02,  4.92858732e-02,  7.32800789e-03,  2.16665080e-01,
        6.54230012e-01,  0.00000000e+00,  3.33666667e-03,  3.23191679e-02,
        2.11124944e-03,  2.71117168e-02,  1.26715522e-02,  3.76772208e-02,
        1.26702344e-02,  3.76706168e-02,  2.51939438e-03,  2.75197527e-02,
        0.00000000e+00,  0.00000000e+00,  7.60635793e-02,  8.34823607e-01,
        2.28675005e-02,  5.33942033e-02,  7.26996557e-03,  3.29478805e-02,
        1.62790698e-01,  1.62790698e-01,  4.65116279e-02,  4.65116279e-02,
        0.00000000e+00,  0.00000000e+00,  3.48837209e-01,  3.48837209e-01,
        2.09302326e-01,  2.09302326e-01,  9.30232558e-02,  9.30232558e-02,
        2.79069767e-01,  2.79069767e-01,  3.95348837e-01,  3.95348837e-01,
        2.55813953e-01,  2.55813953e-01,  1.39534884e-01,  1.39534884e-01,
        3.25581395e-01,  3.25581395e-01,  4.41860465e-01,  4.41860465e-01,
        4.88372093e-01,  4.88372093e-01,  3.72093023e-01,  3.72093023e-01,
        5.58139535e-01,  0.00000000e+00,  6.74418605e-01,  6.74418605e-01,
        5.34883721e-01,  5.34883721e-01,  4.18604651e-01,  4.18604651e-01,
        6.04651163e-01,  6.04651163e-01,  7.20930233e-01,  7.20930233e-01,
        0.00000000e+00,  0.00000000e+00,  4.65116279e-01,  4.65116279e-01,
        6.51162791e-01,  6.51162791e-01,  7.67441860e-01,  7.67441860e-01,
        2.16666667e-01,  5.00000000e-02,  5.00000000e-02,  1.50000000e-01,
        2.00000000e-01,  1.16666667e-01,  0.00000000e+00,  0.00000000e+00,
        0.00000000e+00,  0.00000000e+00,  1.50000000e-01,  6.66666667e-02,
        3.33333333e-02,  1.16666667e-01,  2.00000000e-01,  3.66666667e-01,
        3.16666667e-01,  3.83333333e-01,  3.83333333e-01,  4.00000000e-01,
        5.00000000e-02,  1.00000000e-01,  3.33333333e-02,  5.00000000e-02,
        5.00000000e-02,  5.00000000e-02,  0.00000000e+00,  0.00000000e+00,
        8.33333333e-02,  4.66666667e-01,  0.00000000e+00,  1.66666667e-02,
        1.00000000e-01,  2.50000000e-01,  1.48685178e-01,  3.35974811e-01,
        3.96160249e-01,  1.47097783e-01,  1.35245582e-01,  8.15259013e-02,
        0.00000000e+00,  0.00000000e+00,  0.00000000e+00,  0.00000000e+00,
        7.66484153e-02,  2.23485215e-01,  3.20642919e-01,  2.02716680e-01,
        8.04653056e-02,  5.25515315e-02,  1.09691206e-01,  1.10236476e-01,
        1.01870119e-01,  4.05163967e-02,  2.68243040e-01,  1.20740103e-01,
        3.62756800e-01,  4.28571429e-01,  1.12093114e-01,  4.19048617e-01,
        0.00000000e+00,  0.00000000e+00,  7.41196468e-02,  7.77994982e-02,
        0.00000000e+00,  1.32984023e-01,  7.19981731e-02,  6.87010532e-02,
        3.00000000e+00,  2.00000000e+00,  1.00000000e+00,  0.00000000e+00,
        2.00000000e+00,  0.00000000e+00,  0.00000000e+00,  1.00000000e+00,
        0.00000000e+00,  0.00000000e+00,  1.00000000e+00,  0.00000000e+00,
        0.00000000e+00,  1.00000000e+00,  1.00000000e+00,  1.00000000e+00,
        1.00000000e+00,  1.00000000e+00,  1.16638555e-01,  8.83361445e-01,
        1.00000000e+00, -2.55643238e-02,  8.07981492e-04,  1.32984023e-01,
        8.67015977e-01,  1.00000000e+00,  1.32984023e-01,  1.00000000e+00,
        1.12093114e-01,  8.87906886e-01,  1.00000000e+00,  5.97181930e-02,
        7.93713410e-04,  7.59772096e-02,  9.24022790e-01,  1.00000000e+00,
       -6.09564253e-03,  6.22668978e-04, -2.86000000e+01,  2.96000000e+01,
        1.00000000e+00, -2.86000000e+01,  1.00000000e+00,  7.60705448e-06,
        9.99992393e-01,  1.00000000e+00, -6.89306742e-03,  2.95404526e-04,
        3.24158843e-01,  6.75841157e-01,  1.00000000e+00,  7.03184952e-03,
        2.02939054e-03,  0.00000000e+00,  0.00000000e+00,  0.00000000e+00,
        0.00000000e+00,  0.00000000e+00,  0.00000000e+00,  0.00000000e+00,
        0.00000000e+00,  0.00000000e+00,  0.00000000e+00,  0.00000000e+00,
        0.00000000e+00,  0.00000000e+00,  0.00000000e+00,  0.00000000e+00]))

Issue Analytics

State:
Created 3 years ago
Comments:5

Top GitHub Comments

2reactions

anjackqcommented, Jul 1, 2020

Hi, I got this problem too. For solving it, I just set the upper and lower bound of the box to (1000,-1000) instead of (0,1), and then the simulation works. You can have a try.

0reactions

17150934commented, Jul 28, 2021

finally I find how to

check the file under flow/flow/envs/merge.py or some others. There is a function named

@Property def observation_space(self): “”“See class definition.”“” return Box(low=-1000, high=1000, shape=(5 * self.num_rl, ), dtype=np.float32) set the low to -1000 and the high to 1000

hello,I try this way,but the error above still exists, especially when I set the horizon to a high value(e.g. 1000). So I am very confused