BootDQN+ not matching claimed performance

Several runs on deep_sea/0 i.e., DeepSea with N=10 take longer than 100 episodes, some even longer than 2^10=1024 episodes when running the default_agent BootDQN with no modifications.

To reproduce, this is the code I am running in Colab with a GPU runtime:

# first install bsuite[baselines]
import bsuite
from bsuite.baselines import experiment
from bsuite.baselines.tf import dqn
from bsuite.baselines.tf import boot_dqn

SAVE_PATH_DQN = './logs/test_boot'
env = bsuite.load_and_record("deep_sea/0", save_path=SAVE_PATH_DQN, overwrite=True)
agent = boot_dqn.default_agent(
      obs_spec=env.observation_spec(),
      action_spec=env.action_spec()
)
experiment.run(agent, env, num_episodes=env.bsuite_num_episodes)

I reran this multiple times and have had a few runs with > 1024 bad episodes.

[iosband]

Thanks a lot - I’m seeing the same flavour of results as you in that colab.

Interestingly though I do not see this error in our runs inside Google. My suspicion is that something is going wrong with the versioning/export… but at the moment I don’t understand what that is…

We will try and get to the bottom of this ASAP - thank you for raising!

[iosband]

In fact, I’ve run the colab you linked to and everything is fine now!

Seems like the issue was something to do with tensorflow probability versioning and poor installation in colab