Efficient sampling via DPM-Solver

Hi 👋,

https://github.com/LuChengTHU/dpm-solver just got released (with pytorch code!) and demonstrates how to sample in around 10 steps.
https://arxiv.org/abs/2206.00927

not sure how this compares to the sampling efficiencies that were unlocked with the Elucidated approach. presumably they can’t be combined? but 10 samples is very few; perhaps this would outperform Elucidated?

@article{lu2022dpm,
  title={DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps},
  author={Lu, Cheng and Zhou, Yuhao and Bao, Fan and Chen, Jianfei and Li, Chongxuan and Zhu, Jun},
  journal={arXiv preprint arXiv:2206.00927},
  year={2022}
}

Thanks as always!

[bonlime]

@LuChengTHU hi, i didn’t know where to ping you, so decided to do it here. I really like the quality obtained by the new sampler, but got several questions.

1. automatic111 repo also implements DPM++ (+ Karras scheduling). Do I understand correctly that in order to implement the Karras scheduling with 🤗 diffusers I only need to rewrite set_timesteps function?
2. Previously I’ve used DDIM sampler which allows inversion of image to original noise. Does your multi-step (DPM++) support inverting images? I guess it should as it’s a deterministic sampler but I’m not sure how would the usage of previous step affect the inversion quality. For the DDIM I’ve used equation from original paper (which is exactly the same as for generation process). What would be the equation for DPM++(2M)?

[Birch-san]

if I’m understanding correctly, then yeah @marunine experimented with sampling approaches, including dpm-solver.
hopefully m9 can explain — I don’t wanna misrepresent the conclusions.