Unable to reproduce the results of HER-TD3

When I first run the script with:

python -m examples.her.her_td3_gym_fetch_reach

It raised an error:

Traceback (most recent call last):
  File "/home/fcloud/anaconda3/envs/rlkit/lib/python3.5/runpy.py", line 184, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/fcloud/anaconda3/envs/rlkit/lib/python3.5/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/fcloud/workspace/rlkit/examples/her/her_td3_gym_fetch_reach.py", line 88, in <module>
    experiment(variant)
  File "/home/fcloud/workspace/rlkit/examples/her/her_td3_gym_fetch_reach.py", line 65, in experiment
    **variant['algo_kwargs']
TypeError: __init__() missing 2 required keyword-only arguments: 'her_kwargs' and 'td3_kwargs'

So I modified the source code as follows:

    her_kwargs = dict(observation_key='observation', desired_goal_key='desired_goal')
    td3_kwargs = dict(env=env,
                      qf1=qf1,
                      qf2=qf2,
                      policy=policy,
                      exploration_policy=exploration_policy)
    algorithm = HerTd3(
        her_kwargs=her_kwargs,
        td3_kwargs=td3_kwargs,
        replay_buffer=replay_buffer,
        **variant['algo_kwargs']
    )

Then the experiment can be launched successfully, but the results seem not correct:

[vitchyr]

Split by exp_name or unique_id.

[patrickcho168]

The comments at the top of the HER scripts are confusing (“This should results in an average return of ~3000 by the end of training.”). I think they aren’t meant for the environments in the script?