Learning rate scheduler

Hi, I noticed that with default configuration like:

optimizer = dict(type='SGD', lr=0.005, momentum=0.9, weight_decay=0.0001)
optimizer_config = dict(grad_clip=dict(max_norm=35, norm_type=2))
lr_config = dict(        
    policy='step',       
    warmup='linear',     
    warmup_iters=500,    
    warmup_ratio=1.0 / 3,
    step=[16, 21])    

the learning rate will decrease to 1/10 when training reaches a plateau (several epochs/iters?). Now I want to change those settings, like in pytorch we have ‘patience’ or so. What is the api name for that ? Are there any documentation I can refer to? Thanks!

[leemengwei]

In mmcv/runner/runner.py I saw hook_name = lr_config['policy'].title() + 'LrUpdaterHook' and in mmcv/mmcv/runner/hooks/lr_updater.py I saw things like class FixedLrUpdaterHook(LrUpdaterHook): and class StepLrUpdaterHook(LrUpdaterHook) Thus, learning rate planner in configuration file of mmd should be like lr_config = dict(
policy=‘step’, #Here, could alter to fixed, step, exp, poly, inv, cosine warmup=‘linear’,
warmup_iters=500,
warmup_ratio=1.0 / 3, step=[16, 21])

[ghost]

@rubeea Thanks. After reflection poly schedule seems to be able to help to find the optimal lr. You can fix indirectly start_lr and end_lr by fix min_lr in lr_config = dict(policy='poly', power=0.9, min_lr=1e-4, by_epoch=False), and your initial lr in your optimizer.