Typo for text_encodings?

Hi, me again (lol)

Just curious why set inited text_encodings’s length as 0 https://github.com/lucidrains/DALLE2-pytorch/blob/8b054686530c90ecd8e8db62eb9c648d189accf9/dalle2_pytorch/dalle2_pytorch.py#L747-L748

I tried the test code in the Readme, but throw an error

concated token’s shape: (b 4 d), :[text_encoding(b,0,d), text_embed(b,1,d), time_embed(b,1,d), image_embed(b,1,d), learned_queries(b,1,d)] mask’s shape: (b 5)).

diffusion_prior = DiffusionPrior(
    net = prior_network,
    clip = clip,
    timesteps = 100,
    cond_drop_prob = 0.2,
    condition_on_text_encodings = False  # this probably should be true, but just to get Laion started
).cuda()

But, when I write as text_encodings = torch.empty((batch, 1, dim), device = device, dtype = dtype) , error gone.

Plz have a check. Enjoy!

[CiaoHe]

Yeah, weird, the latest version is cool. My wrong haha (let me check it)

I set the mask wrong mask = torch.ones((batch, text_encodings.shape[-2]), device=device, dtype=torch.bool) (text_encodings.shape[-2], I wrote as text_embeds.shape[-2])

[CiaoHe]

@CiaoHe wow, you retyped all ~2k lines of code? 🤯

Sounds really funny haha. But I think it better to check than just fork the whole down