ONNX compatibility issues

Thanks for the great library!

I’m looking into generating ONNX version of the stable diffusion models.

I had to make few changes to unet_2d_condition.py to get it to work. You can find some of the changes here… https://github.com/harishanand95/diffusers/commit/8dd4e822f87e1b4259755a2181218797ceecc410 Let me know if you prefer a PR for this and if there exist any implementations for return non dict that I can refer to. Thanks!

1. reason for removing broadcast_to :

torch.onnx.symbolic_registry.UnsupportedOperatorError: Exporting the operator ::broadcast_to to ONNX opset version 16 is not supported. Please feel free to request support or submit a pull request on PyTorch GitHub.

1. reason for dict changes:

RuntimeError: Encountering a dict at the output of the tracer might cause the trace to be incorrect, this is only valid if the container structure does not change based on the module's inputs. Consider using a constant container instead (e.g. for list, use a tuple instead. for dict, use a NamedTuple instead). If you absolutely need this and know the side effects, pass strict=False to trace() to allow this behavior. 

I was able to generate images with onnxruntime and DirectML, there are some more issues to be fixed.

• Instructions: https://github.com/harishanand95/diffusers/blob/dml/examples/inference/readme.md#instructions-for-onnx-dml-execution-on-windows

• A script that creates onnx files https://github.com/harishanand95/diffusers/blob/dml/examples/inference/save_onnx.py

• A StableDiffusionPipeline implementation that runs on onnxruntime https://github.com/harishanand95/diffusers/blob/dml/examples/inference/dml_onnx.py

Let me know if you have further concerns or ideas, thanks!

[pcuenca]

You are right, @harishanand95, tile does not work with ONNX. I’ll open a PR with your approach then. Thanks a lot!

[harishanand95]

Thanks!