Only using 65% of GPU memory

While running some benchmarks I noticed min-dalle was only using 65% of my 3070 Ti GPU’s memory:

Here’s the function I’m calling:

def run_dalle():
    generate_image(
        is_mega=True,
        text='rich ducks playing poker',
        seed=0,
        grid_size=3,
        top_k=256,
        image_path='generated',
        models_root='pretrained',
        fp16=True,
    )

After running it 50 times, each image takes 54.5 seconds on average to generate. It’s running baremetal on 32x E5-2630 v3 threads and 64 GB RAM.

Is there a way to make it use more of the GPU? or am I reading this wrong? Thanks!

[78Alpha]

I had been using my own utility script for batch generation here

I have it set to is_mega=True

[78Alpha]

@78Alpha are you using the Mega model?

The default for the pip package

EDIT:

Checking back, it is the Mega Version. Will also try non-mega…

Non-mega went to about 9 seconds per image