Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[Community Pipeline] Speech to Image

See original GitHub issue

Intro

Community Pipelines are introduced in diffusers==0.4.0 with the idea of allowing the community to quickly add, integrate, and share their custom pipelines on top of diffusers.

You can find a guide about Community Pipelines here. You can also find all the community examples under examples/community/. If you have questions about the Community Pipelines feature, please head to the parent issue.

Idea: Speech to Image

You can use a transformer automatic-speech-recognition such as OpenAI whisper to transcribe the text, and pass that to Stable Diffusion. Together, this would create a nice speech-to-image pipeline.

Resources

Issue Analytics

State:
Created a year ago
Comments:9 (6 by maintainers)

Top GitHub Comments

2reactions

fffilonicommented, Oct 25, 2022

📌 gradio demo is available here: https://huggingface.co/spaces/fffiloni/speech-to-image

1reaction

MikailINTechcommented, Oct 19, 2022

Hi @osanseviero, I opened a PR for this issue. For the moment, I have tested my pipeline for whisper and it works pretty well. I’m waiting for your suggestions (maybe making the pipeline more generic so that we can use other speech-recognition models ?)

Top Results From Across the Web

Speech to Diffusion • Community Pipeline - Hugging Face

Speech to Diffusion • Community Pipeline. This demo can generate an image from an audio sample using pre-trained OpenAI whisper-small and Stable Diffusion....

Speech-to-Image Creation Using Jina | Jina AI

Use Jina, Whisper and Stable Diffusion to build a cloud-native application for generating images using your speech.

[ML-News] Speech To Image • Community pipeline - a Hugging ...

fffiloni「We @_akhaliq and i just finished to build a @Gradio demo on @huggingface for Speech to image community pipeline which was initially proposed...

Image2speech: Automatically generating audio descriptions of ...

Experiments reported in this paper convert image feature vectors into speech unit sequences. In order to implement this pipeline, four types of standard ......

ASR and TTS API Deployment Guide - Vakyansh

Jenkins can be set up optionally as a deployment pipeline of CI/CD and ... True --image-name gcr.io/ekstepspeechrecognition/speech_recognition_model_api ...