Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

numpy_to_pil fails on flax pipeline output

See original GitHub issue

Describe the bug

When I try to numpy_to_pil the output of FlaxStableDiffusionPipeline, it crashes with

‘DeviceArray’ object has no attribute ‘array_interface’

(maybe possibly related to safety_checker=None)

Reproduction

pipe, params = diffusers.FlaxStableDiffusionPipeline.from_pretrained(
    "CompVis/stable-diffusion-v1-4",
    revision="flax",
    safety_checker=None,
    device_map='auto',
    dtype=jnp.float16,
)

prompt="A delightful child enjoying a hearty bowl of JAX cereal."
prompt_inputs = pipe.prepare_inputs(prompt)
result = pipe(
    prompt_ids=prompt_inputs, 
    params=params,
    num_inference_steps=12,
    prng_seed=jax.random.PRNGKey(0)
)

images = pipe.numpy_to_pil(result.images)

Logs

diffusers/pipeline_flax_utils.py:447 in <listcomp>                                                          
│   444 │   │   if images.ndim == 3:                                                               
│   445 │   │   │   images = images[None, ...]                                                     
│   446 │   │   images = (images * 255).round().astype("uint8")                                    
│ ❱ 447 │   │   pil_images = [Image.fromarray(image) for image in images]                          
│   448 │   │                                                                                      
│   449 │   │   return pil_images                                          

Image.py:2803 in fromarray
│ ❱ 2803 │   arr = obj.__array_interface__                                                         
AttributeError: 'DeviceArray' object has no attribute '__array_interface__'

System Info

diffusers version: 0.5.1
Platform: Linux-5.15.0-50-generic-x86_64-with-glibc2.35
Python version: 3.10.6
PyTorch version (GPU?): 1.12.1+cu116 (True)
Huggingface_hub version: 0.10.1
Transformers version: 4.23.1
Using GPU in script?: unsure? CPU utilization is high, vRAM allocation is high, GPU utilization is low. result.images.device() is CPU.
Using distributed or parallel set-up in script?: no

Issue Analytics

State:
Created a year ago
Comments:8 (7 by maintainers)

Top GitHub Comments

1reaction

pcuencacommented, Oct 27, 2022

Assigning to myself to take a look soon.

1reaction

patrickvonplatencommented, Oct 14, 2022

I’d advocate to just return numpy images only in Flax no matter what -> it’s different from PyTorch because we want to be able to compile the whole function end-to-end which is not possible with PIL

Top Results From Across the Web

Error when transforming numpy array to PIL Image

I want to convert the array to a PIL image. To do so I call img = Image.fromarray(img, mode='L') but the output I...

Releases · huggingface/diffusers - GitHub

import torch from diffusers import UnCLIPPipeline pipe ... Ensure Flax pipeline always returns numpy array by @pcuenca in #1435; Add 2nd order heun ......

CLIP - Hugging Face

CLIP is a multi-modal vision and language model. It can be used for image-text similarity and for zero-shot image classification. CLIP uses a...

diffusers - PyPI

import torch from diffusers import StableDiffusionPipeline pipe ... import jax import numpy as np from flax.jax_utils import replicate from ...

Numpy Reader — NVIDIA DALI 1.20.0 documentation

By default, out-of-bounds access results in an error, but it can be controlled by the out_of_bounds_policy argument. Let us see an example of...