Weirdly bad results with the example scripts

Hi, I’ve been running some of the experiment scripts provided with the package and I am getting weirdly bad results – mostly there is just no visible improvement at all. E.g. when running:

python experiments/dqn_2way-single-intersection.py

and plotting results using

python outputs/plot.py -f outputs/2way-single-intersection/dqn

I get the following results:

Are you getting the same results? Is this the expected behaviour with this experiment? Is it just that the hyperparameters in the script are really bad, the scenario is challenging, or is it that something in my setup is not working correctly?

As a suggestion – maybe we should run all the included experiments scripts and store the results somewhere for reference. That would enable users to make sure that everything works as intended before they start running their own experiments.

[LucasAlegre]

Do these plots correspond to the training phase? and why they don’t have a curve shape decreasing over time? how we can explain these picks? do they relate to different hours? I hope my questions make sense.

Yes, they are generated during the training. They generally have, it depends on how fast the agent learns a good policy. There are still stochastic behavior duo to the vehicles behaviors and the flow defined in the route file.

[LucasAlegre]

Hey, I would like to ask, how I can plot only specific episodes? And by episode do you mean an episode on the simulator, i.e 100000 steps?

The environment outputs a file for every run separately, so you can plot a given run with (e.g. run 1): python outputs/plot.py -f outputs/2way-single-intersection/a3c_conn0_run1.csv By episode I mean a “run” on the simulator until the simulation ends.