CosineAnnealingWarmRestarts is not compatible with LRScheduler.simulate_values()

🐛 Bug description

CosineAnnealingWarmRestarts is not compatible with LRScheduler.simulate_values(). Precisely, a object of type CosineAnnealingWarmRestarts can’t be replicated by LRScheduler._replicate_lr_scheduler(). It works for CosineAnnealingLR.

For example :

lr_scheduler = CosineAnnealingWarmRestarts(optimizer=optimizer, T_0=10, eta_min=1e-3)
lr_values = LRScheduler.simulate_values(num_events=50, lr_scheduler=lr_scheduler)

produces the following error

Traceback (most recent call last):
  File "tutorials/misc/lr_schedulers.py", line 56, in <module>
    lr_values = LRScheduler.simulate_values(num_events=50, lr_scheduler=lr_scheduler)
  File "/work/desrozis/Softwares/conda/envs/focus-light/lib/python3.7/site-packages/ignite/contrib/handlers/param_scheduler.py", line 606, in simulate_values
    copy_lr_scheduler = LRScheduler._replicate_lr_scheduler(lr_scheduler)
  File "/work/desrozis/Softwares/conda/envs/focus-light/lib/python3.7/site-packages/ignite/contrib/handlers/param_scheduler.py", line 627, in _replicate_lr_scheduler
    copy_lr_scheduler = lr_scheduler_cls(optimizer=dummy_optimizer, **kwargs)
TypeError: __init__() got an unexpected keyword argument 'T_i'

This issue is due to the assumption that lr_scheduler.__state_dict__ contains arguments of method CosineAnnealingWarmRestarts.__init__(). A workaround could be to remove T_i in a similar way to base_lrs and last_epoch but it’s not very satisfactory…

Hope it helps to have a better and stonger ignite.

Environment

• PyTorch Version (e.g., 1.4): 1.4
• Ignite Version (e.g., 0.3.0): 0.3.0
• OS (e.g., Linux): Linux
• How you installed Ignite (conda, pip, source): conda
• Python version: 3.7.6
• Any other relevant information:

[sdesrozis]

You save my day 😊 I check if it’s ok.

[vfdev-5]

@sdesrozis how about the same approach as used with lr finders etc :

• save initial state dicts
• do something
• restore init state ?