Speech to image pipeline, Unexpected output, green image

Describe the bug

Resuting image is greenish

Reproduction

import torch

import matplotlib.pyplot as plt
from datasets import load_dataset
from diffusers import DiffusionPipeline
from transformers import (
    WhisperForConditionalGeneration,
    WhisperProcessor,
)


device = "cuda" if torch.cuda.is_available() else "cpu"

ds = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation")

audio_sample = ds[3]

text = audio_sample["text"].lower()
speech_data = audio_sample["audio"]["array"]

model = WhisperForConditionalGeneration.from_pretrained("openai/whisper-small").to(device)
processor = WhisperProcessor.from_pretrained("openai/whisper-small")

diffuser_pipeline = DiffusionPipeline.from_pretrained(
    "CompVis/stable-diffusion-v1-4",
    custom_pipeline="speech_to_image_diffusion",
    speech_model=model,
    speech_processor=processor,
    revision="fp16",
    torch_dtype=torch.float16,
)

diffuser_pipeline.enable_attention_slicing()
diffuser_pipeline = diffuser_pipeline.to(device)

output = diffuser_pipeline(speech_data)
plt.imsave('aa.png',output.images[0])

the results seems to be misaligned image

Logs

No response

System Info

• diffusers version: 0.6.0
• Platform: Linux-4.15.0-142-generic-x86_64-with-glibc2.23
• Python version: 3.9.13
• PyTorch version (GPU?): 1.8.1+cu101 (True)
• Huggingface_hub version: 0.10.1
• Transformers version: 4.23.1
• Using GPU in script?: yes
• Using distributed or parallel set-up in script?:No

[MikailINTech]

@MikailINTech do you mind taking a look here? 😃

I’ll have a look at it, thanks for tagging me

[darwinharianto]

Sorry, I found it I wasn’t supposed to plt.imsave('aa.png',output.images[0]) but output.images[0].save('bb.png')