Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Dreambooth: cosine_with_restarts learning rate schedule won't restart

See original GitHub issue

Describe the bug

Hello,

During log observation of a three dreambooth training logs found that `–lr_scheduler=“cosine_with_restarts” lead to a cosine-alike lr change on a tensorboard.

Reproduction

Training finishes successfully and results are pretty good

Logs

No response

System Info

diffusers==0.6.0.dev0

Issue Analytics

State:
Created 10 months ago
Comments:7 (6 by maintainers)

Top GitHub Comments

1reaction

pcuencacommented, Dec 13, 2022

That’d be amazing @zetyquickly! Maybe we can include the polynomial power too?

0reactions

zetyquicklycommented, Dec 13, 2022

I can make a PR, if it’s not in your top priority list to do

Top Results From Across the Web

Cosine annealed warm restart learning schedulers - Kaggle

The cosine annealed warm restart learning schedule has two parts, cosine annealing and warm restarts. Cosine annealing means that the cosine function is ......

Training Stable Diffusion with Dreambooth using Diffusers

We recommend using a low learning rate and progressively increasing the number of steps until the results are satisfactory. Dreambooth needs ...

CosineDecayRestarts - Keras

This schedule applies a cosine decay function with restarts to an optimizer step, given a provided initial learning rate. It requires a step...

Use tensorflow learning-rate decay in a Keras-to-TPU model

I like to use cosine restart learning rate decay when I fit my models. I've coded up my own as a keras callback,...

g/ - /sdg/ - Stable Diffusion General - Technology - 4Chan

The point of using SD model was its large training data set. ... into models dierctor, restarted... I see nothing. I'm also running...