[Question] Callback for GAIL
See original GitHub issueHello!
I am training GAIL
on Pendulum-v0
. The expert data for Pendulum-v0
is available in expert_path
. I am trying to save the model using the Callback documentation as reference. Here’s the code:
import gym
import tensorflow as tf
from stable_baselines import GAIL, SAC, TRPO, PPO2
from stable_baselines.gail import ExpertDataset, generate_expert_traj
from stable_baselines.common.callbacks import CheckpointCallback, EvalCallback
tf.compat.v1.logging.set_verbosity(tf.compat.v1.logging.ERROR)
if __name__=='__main__':
expert_path = 'stable_baselines/gail/dataset/expert_pendulum.npz'
dataset = ExpertDataset(expert_path=expert_path, traj_limitation=10, verbose=0)
model = GAIL('MlpPolicy', 'Pendulum-v0', dataset, verbose=0)
checkpoint_callback = CheckpointCallback(save_freq=1000, save_path='./logs/', name_prefix='gail')
model.learn(total_timesteps=2000, callback=checkpoint_callback)
Here’s the traceback:
Traceback (most recent call last):
File "checking_callback.py", line 18, in <module>
main()
File "checking_callback.py", line 15, in main
model.learn(total_timesteps=2000, callback=checkpoint_callback)
File "C:\IL_UAV\GAIL-TF-SB\stable_baselines\gail\model.py", line 54, in learn
return super().learn(total_timesteps, callback, log_interval, tb_log_name, reset_num_timesteps)
File "C:\IL_UAV\GAIL-TF-SB\stable_baselines\trpo_mpi\trpo_mpi.py", line 455, in learn
ob_expert, ac_expert = self.expert_dataset.get_next_batch()
File "C:\IL_UAV\GAIL-TF-SB\stable_baselines\gail\dataset\dataset.py", line 151, in get_next_batch
if dataloader.process is None:
AttributeError: 'NoneType' object has no attribute 'process'
Some comments and queries:
- The callback runs without errors if GAIL is replaced with an RL algorithm (i.e. removing the
dataset
parameter & replacing GAIL with TRPO, SAC, PPO2, etc.) I believe the error has to do with the way CheckpointCallback’ssave_path
parameter works specifically for GAIL. - I ran the same code with EvalCallback. The result is the same, and it turns out the error has to do with
best_model_save_path
(remove this parameter and the code runs fine). - Should I be using CustomCallback and
rollout
s instead? If yes, could you give me a specific example code? Thanks!
PS: I am new to both Stable Baselines and RL
System Info
- Windows 10
- Python 3.7.0
- Tensorflow 1.14.0
Issue Analytics
- State:
- Created 3 years ago
- Comments:16
Top Results From Across the Web
GAIL Interview Questions (2022) | Glassdoor
GAIL interview details: 68 interview questions and 61 interview reviews posted anonymously by GAIL interview candidates.
Read more >Callback functions - Vanilla Embed Library - Typeform's APIs
The onQuestionChanged callback will execute whenever respondent navigates between questions. The event is sent when navigating in the form forward or backward.
Read more >Breast Cancer Risk Assessment and Screening in Average ...
False-positive test results from mammography include callbacks for additional images and follow-up biopsies that are found to be benign. The U.S. Preventive ...
Read more >The Callback Queen (2013) - IMDb
Gail Brady and Drew Carter-Cain in a still from 'The Callback Queen' ... Questions were also received by Amy-Joyce Hastings. She was charming....
Read more >Callbacks — Stable Baselines 2.10.3a0 documentation
To build a custom callback, you need to create a class that derives from BaseCallback . This will give you access to events...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
You need to customize SB a bit and pass the callback in https://github.com/hill-a/stable-baselines/blob/c4c31cb5687800cf102054f0935aa09e7545159f/stable_baselines/common/runners.py#L58
We don’t do that because this would break the code if users use MPI with more than 1 node.
You can also change
timesteps_per_batch
but this not recommended if it deteriorates the performance.https://github.com/hill-a/stable-baselines/blob/c4c31cb5687800cf102054f0935aa09e7545159f/stable_baselines/trpo_mpi/trpo_mpi.py#L50
#847 solves both issues as they were related to the same problem (continuing training after saving).