set global_step from checkpoint

❓ Questions/Help/Support

Hello, I wonder how am I suppose to set the global_step from checkpoint for handlers?

Specifically, i would like to have most of the handlers in tensorboard logger using engine.state.iteration as global step. Since engine.state.iteration is reset when i do engine.run. I had to create my own counter (like the following), which basically add an offset to current iteration, and pass it to all the global_step_transform arguments in output handler.

class global_step_counter:
    def __init__(self, resume_from_iter):
        self.resume_from_iter = resume_from_iter
        return 
    def set_starting_val(self, new_val):
        self.resume_from_iter = new_val
        return 
    def __call__(self, engine):
        return engine.state.iteration + self.resume_from_iter

However, i realize the following handlers does not provide global_step_transform option

• OptimizerParamsHandler
• WeightsScalarHandler
• GradsScalarHandler
• WeightsHistHandler
• GradsHistHandler Instead, they use get_event_attrib_value to get the global_step, which would make this variable inconsistent with others

Thank you

[ghost]

Thank you @vfdev-5 ! this solved my issue

[ghost]

Here is a minimal example @vfdev-5

from collections import OrderedDict
import numpy as np
from ignite.engine import Engine
init_state = OrderedDict([('seed', 12), ('epoch_length', 8), ('max_epochs', 1), ('iteration', 8)])

def dataset():
    yield np.random.random()

e = Engine(lambda x , b: b)
e.load_state_dict(init_state)
print(f'e.state before run: {e.state}')
e.run(dataset(), epoch_length=1)
print(f'e.state after run: {e.state}')

output

(.venv)  ◰³ .venv  ~/micro  python test.py                                                                                                              Wed Mar  4 00:01:33 2020
e.state before run: State:
	iteration: 8
	epoch: 1
	epoch_length: 8
	max_epochs: 1
	output: <class 'NoneType'>
	batch: <class 'NoneType'>
	metrics: <class 'dict'>
	dataloader: <class 'NoneType'>
	seed: 12

e.state after run: State:
	iteration: 1
	epoch: 1
	epoch_length: 1
	max_epochs: 1
	output: 0.15416284237967237
	batch: 0.15416284237967237
	metrics: <class 'dict'>
	dataloader: <class 'generator'>
	seed: 12

You can see the iteration is indeed reset