Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[Community] FP16 ONNX produces incorrect output

See original GitHub issue

Describe the bug

#932 enabled conversion of the main branch FP32 model (git clone https://huggingface.co/CompVis/stable-diffusion-v1-4 -b main) to ONNX FP16. While it runs fine with OnnxStableDiffusionPipeline using DMLExecutionProvider (onnxruntime-directml==1.13.1), the produced image is just a black square.

Conversion was done with: convert_stable_diffusion_checkpoint_to_onnx.py --model_path stable-diffusion-v1-4 --output_path fp16_sd14 --fp16

Reproduction

from diffusers import OnnxStableDiffusionPipeline

pipe = OnnxStableDiffusionPipeline.from_pretrained(“fp16_sd14”, provider=“DmlExecutionProvider”) prompt = “viking storming a castle” image = pipe(prompt).images[0] image.save(“viking.png”)

Logs

No response

System Info

diffusers version: 0.7.0.dev0
Platform: Windows-10-10.0.22000-SP0
Python version: 3.9.13
PyTorch version (GPU?): 1.12.1+cpu (False)
Huggingface_hub version: 0.10.0
Transformers version: 4.22.2
Using GPU in script?: Yes
Using distributed or parallel set-up in script?: No

Issue Analytics

State:
Created a year ago
Comments:8 (3 by maintainers)

Top GitHub Comments

1reaction

anton-lcommented, Nov 8, 2022

https://github.com/huggingface/diffusers/pull/1173 didn’t help, keeping the issue open. Can’t reproduce at the moment without a DirectML setup (works well on cuda and cpu though).

If anyone reading this has a hunch about what could be wrong, feel free to open a bugfix PR!

0reactions

sudo-carsoncommented, Nov 14, 2022

I’ve been converting models via Colab as my system doesn’t have enough VRAM, and interestingly, testing a float16 model on Colab yields correct results, but using the model on my system yields NaN. Could this issue be related to this comment then?

Top Results From Across the Web

(optional) Exporting a Model from PyTorch to ONNX and ...

To export a model, we call the torch.onnx.export() function. This will execute the model, recording a trace of what operators are used to...

Tune performance | onnxruntime

ONNX Runtime provides high performance across a range of hardware options through its Execution Providers interface for different execution environments.

Speeding Up Deep Learning Inference Using TensorFlow ...

In this post, you learn how to deploy TensorFlow trained deep learning models using the new TensorFlow-ONNX-TensorRT workflow.

Solved: option of mo.py "--data_type FP16 " - Intel Communities

The INT8 IR will be produced automatically if you supply an INT8 ONNX as input. ... model with the data_type option "FP16", which...

Pytorch to ONNX: Could not find an implementation for ...

For anyone trying to replicate this problem, I have made a minimal example. ... Running the above code results in the below error:...