Stable Diffusion 2 and Apple Silicon/MPS

Describe the bug

The diffusers code will run under Apple Silicon/MPS for Stable Diffusion 2.0 but the result is always noise. No images are generated.

Additionally, if you specify pipe.enable_attention_slicing(), then you get the following error — this is with the latest, dev source from the site and is not using the stable release.

│ /Users/fahim/Code/Python/sd2/diff.py:11 in <module>                                              │
│                                                                                                  │
│    8 device = torch.device("cuda" if torch.cuda.is_available() else "mps" if torch.has_mps el    │
│    9 scheduler = EulerDiscreteScheduler.from_pretrained(model_id, subfolder="scheduler")         │
│   10 pipe = StableDiffusionPipeline.from_pretrained(model_id, scheduler=scheduler, safety_che    │
│ ❱ 11 pipe.enable_attention_slicing()                                                             │
│   12 # pipe = StableDiffusionPipeline.from_pretrained(model_id).to(device)                       │
│   13                                                                                             │
│   14 prompt = "a photo of an astronaut riding a tricerotops"                                     │
│                                                                                                  │
│ /Users/fahim/miniconda3/envs/ml/lib/python3.9/site-packages/diffusers/pipelines/stable_diffusion │
│ /pipeline_stable_diffusion.py:170 in enable_attention_slicing                                    │
│                                                                                                  │
│   167 │   │   if slice_size == "auto":                                                           │
│   168 │   │   │   # half the attention head size is usually a good trade-off between             │
│   169 │   │   │   # speed and memory                                                             │
│ ❱ 170 │   │   │   slice_size = self.unet.config.attention_head_dim // 2                          │
│   171 │   │   self.unet.set_attention_slice(slice_size)                                          │
│   172 │                                                                                          │
│   173 │   def disable_attention_slicing(self):                                                   │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
TypeError: unsupported operand type(s) for //: 'list' and 'int'

Reproduction

This is the script used:

import torch
from diffusers import StableDiffusionPipeline, EulerDiscreteScheduler

model_id = "stabilityai/stable-diffusion-2"

# Use the Euler scheduler here instead
device = torch.device("cuda" if torch.cuda.is_available() else "mps" if torch.has_mps else "cpu")
scheduler = EulerDiscreteScheduler.from_pretrained(model_id, subfolder="scheduler")
pipe = StableDiffusionPipeline.from_pretrained(model_id, scheduler=scheduler, safety_checker=None).to(device)

prompt = "a photo of an astronaut riding a tricerotops"
image = pipe(prompt, height=512, width=512, num_inference_steps=25).images[0]

image.save("test.png")

Logs

No response

System Info

• diffusers version: 0.9.0.dev0
• Platform: macOS-13.0-arm64-arm-64bit
• Python version: 3.9.13
• PyTorch version (GPU?): 1.14.0.dev20221124 (False)
• Huggingface_hub version: 0.10.1
• Transformers version: 4.24.0
• Using GPU in script?: yes. See script
• Using distributed or parallel set-up in script?: no. See script

[lkaupp]

any idea when this fix is also available in the inpainting repository? Edit:// The other repo is a bunch of fixes away; everything beyond 512x512 px is not doable with my RTX 3090 24 GB memory… xformers and slicing (manually c&p from this fix pasted) working.

[FahimF]

After testing today, I can confirm that this works on my Apple M1 🙂 Thank you!