Is the use of `torch.manual_seed` in example training code correct?

Describe the bug

https://github.com/huggingface/diffusers/blob/92b6dbba1a25ed27b0bae38b089715132c7e6bcc/examples/train_unconditional.py#L168

This is a minor thing, but I think this should be torch.Generator().manual_seed(0). In my understanding, if torch.manual_seed is called, it sets the seed globally and could cause unexpected side effect. I think it’s better not to change the global seed in the training loop.

Reproduction

No response

Logs

No response

System Info

diffusers==0.1.3 (current `main` branch `92b6dbba1a` too)

[leopoldmaillard]

Hello @patrickvonplaten ! It seems that torch.manual_seed(0) is still called to set the generator in the unconditional training script.

I noticed this when tweaking the script to be able to resume a training process from a checkpoint. My shuffled Dataloader produced the same mini batches in the same order from one training to another.

[aweinmann]

related to this issue, pipelines do not pass the generator to the step function (see: https://github.com/huggingface/diffusers/blob/bfe37f31592a8fa4780833bf4e7fbe18fa9f866c/src/diffusers/pipelines/ddpm/pipeline_ddpm.py#L61) resulting in different evaluation outputs when not resetting the default generator. while the initial noise will use the passed in generator subsequent added noise in the step function will not.