How to use TimeMetricsHook in a train task?

❓ Questions and Help

I want use TimeMetricsHook to get the training performance, but I face the following error:

Traceback (most recent call last):
  File "main.py", line 255, in <module>
    main()
  File "main.py", line 111, in main
    train(datasets, model, loss, optimizer, meters, args)
  File "main.py", line 145, in train
    trainer.train(task)
  File "/home/xiaobinz/anaconda3/envs/pytorch-bw/lib/python3.6/site-packages/classy_vision/trainer/classy_trainer.py", line 78, in train
    task.step(self.use_gpu)
  File "/home/xiaobinz/anaconda3/envs/pytorch-bw/lib/python3.6/site-packages/classy_vision/tasks/classy_task.py", line 178, in step
    hook.on_step(self)
  File "/home/xiaobinz/anaconda3/envs/pytorch-bw/lib/python3.6/site-packages/classy_vision/hooks/time_metrics_hook.py", line 50, in on_step
    self._log_performance_metrics(task)
  File "/home/xiaobinz/anaconda3/envs/pytorch-bw/lib/python3.6/site-packages/classy_vision/hooks/time_metrics_hook.py", line 84, in _log_performance_metrics
    get_rank(), task.perf_stats.report_str()
  File "/home/xiaobinz/anaconda3/envs/pytorch-bw/lib/python3.6/site-packages/classy_vision/generic/perf_stats.py", line 217, in report_str
    name_width = max(len(k) for k in self._host_stats.keys())
ValueError: max() arg is an empty sequence

The following code is how do I use TimeMetricsHook in my model:

def train(datasets, model, loss, optimizer, meters, args):
     task = (ClassificationTask()
             .set_num_epochs(args.num_epochs)
             .set_loss(loss)
             .set_model(model)
             .set_optimizer(optimizer)
             .set_meters(meters))
     for phase in ["train", "test"]:
         task.set_dataset(datasets[phase], phase)

     hooks = [LossLrMeterLoggingHook(log_freq=args.print_freq)]
     # show progress
     hooks.append(ProgressBarHook())
     # show time bar
     hooks.append(TimeMetricsHook(log_freq=args.print_freq))

     checkpoint_dir = f"{args.video_dir}/checkpoint/classy_checkpoint_{time.time()}"
     os.mkdir(checkpoint_dir)
     hooks.append(CheckpointHook(checkpoint_dir, input_args={}))

     task = task.set_hooks(hooks)
     trainer = LocalTrainer(use_gpu = args.cuda)
     trainer.train(task)

The LossLrMeterLoggingHook , ProgressBarHook, and CheckpointHook can works, but add TimeMetricsHook, will get the error.

[mannatsingh]

Yes, the TimeMetricsHook doesn’t work with ClassificationTask anymore. I think it was preserved for when users wanted to add the perf stats logic in the task for debugging. @vreis maybe we can remove the hook / add a warning to the docs if we still want to retain that hook.

@XiaobingSuper to get the training performance, you can just use the TensorboardPlotHook. We plot the training time per epoch. A sample run looks like this -

[XiaobingSuper]

Thanks for all your help, now, I will try to use TensorboardPlotHook to get the training performance.