Possibly broken unet2+

It seems that unet1 progresses quickly. On my dataset, within 10-20 epochs I can get pretty good results at 64px:

But when trying unet2, the results are poor even after more than 20-40 epochs:

Here’s another example of unet2 of mel-spectograms after many many steps:

Granted, it is possible that unet1 results also look similar, but it’s hard to tell with the blur going on. I assume that’s the result of the noising function? Is there a way to turn off the noising functionality during sample if stop_at_unet is specified at 1? That said, with the spectogram, unet1 produces very good blacks in the padding area, but unet2 can’t get it right.

I know other’s have voiced issues with unet2 as well. Here are my unet2 settings for reference:

unet2 = dict(
            dim=128,
            cond_dim=512,
            dim_mults=(1, 2, 3, 4),
            cond_images_channels=cond_images_channels,
            num_resnet_blocks=2,
            layer_attns=(False, False, False, True,
            layer_cross_attns=(True, True, True, True),
            # final_conv_kernel_size=1,
            memory_efficient=True
        )

Note: I’ve also tried with dim_mults (1, 2, 4, 6) and num_resnet_blocks=(2, 2, 4, 8) with similar results.

[deepglugs]

My samples don’t look too bad either after lots of training. I’ve switched back and forth between random cropped unet2 training and full-size unet2 training and I think that has helped. Images aren’t perfect still, but there appears to be progress. Closing for now.

[integer753]

Yes i’m using the non-elucidated imagen at the moment. Here is one of the results i have at the moment, but i need to retrain a little bit because i couldn’t transfer all my layers to the new version, so things are a bit mangled, i’ll post something when i have better results. In any case the upscaler is working for me now!