Choosing the learning rate

Hello @KichangKim !

I have a question about learning rate values used for trainig, especially for v3. As far as I understand the code

"learning_rates": [
  {
      "used_epoch": 0,
      "learning_rate": 5.0
  }
]

# Udpate learning rate
for learning_rate_per_epoch in learning_rates:
    if learning_rate_per_epoch['used_epoch'] <= int(used_epoch):
        learning_rate = learning_rate_per_epoch['learning_rate']

you set learning rate to 5.0 during the first epoch and don’t change it anymore.

Do I understand it correclty? If yes, what is the reason for using such a high learning rate? Most of the ResNets I’ve seen were trained with lr=0.01 or lower.

[yozhikoff]

DeepDanbooru is a great project, be sure that reference is in your pocket already!

And thanks for the help.

[yozhikoff]

I use v3.

I made a custom tag list made only of tags from my database.

Could you drop me an email to yozhikoff [at] protonmail.com? Would be easier to share code there.