What is the network structure to generate mean and std for sampling process in the training phase?

I am trying to reproduce the result of variational autoencoder part of the entire network. Since the training code was not released yet, I can only guess you are doing something like the following code.

I trained with this structure, but found it is really hard to get a descent result. I am wondering if you could provide more training details, such as complete network structure for training, training time, if there are some special training strageties and tricks.

        self.encoder_latent = nn.Sequential(*model)
        self.mu = ResnetBlock(min(ngf * mult * 2, opt.mc),
                              padding_type=padding_type,
                              activation=activation,
                              norm_layer=norm_layer,
                              opt=opt,
                              )
        self.log_var = ResnetBlock(min(ngf * mult * 2, opt.mc),
                                   padding_type=padding_type,
                                   activation=activation,
                                   norm_layer=norm_layer,
                                   opt=opt,
                                   )
        model += [self.mu]

        # model += [nn.Conv2d(min(ngf * mult * 2, opt.mc), min(ngf, opt.mc), 1, 1)]
        if opt.feat_dim > 0:
            # 1*1 conv
            model += [nn.Conv2d(min(ngf * mult * 2, opt.mc), opt.feat_dim, 1, 1)]

        self.encoder = nn.Sequential(*model)

    def train_forward(self, input):
        latent = self.encoder_latent(input)
        mu = self.mu(latent)
        log_var = self.log_var(latent)
        z = self.sample(mu, log_var)
        return self.decoder(z), mu, log_var

    def sample(self, mu, log_var):
        std = torch.exp(log_var * 0.5)
        eps = torch.randn_like(mu)
        return mu + std * eps

[Zeqiang-Lai]

the common practice usually uses fully connected layers

Yes, pytorch VAE examples also use fully connected layers For me, I think using fc layers could save the global info, while using 1*1 conv will only have local receptive field. Correct me if I am wrong

I currently use two 1*1 convolutions as it is suggested in the test code

where is the suggestion in code, I couldnt find it

sorry, my fault, it’s two resnet blocks. The test code directly use mean from the encoder and feed it into the decoder(see Link. So the last layer of the encoder should output mean. As it is shown in the test code, the block to output mean is a resnet block (in testing, feat_dim = -1, so the last 1*1 convolution is not used)

        model += [
            ResnetBlock(
                min(ngf * mult * 2, opt.mc),
                padding_type=padding_type,
                activation=activation,
                norm_layer=norm_layer,
                opt=opt,
            )
        ]
        # model += [nn.Conv2d(min(ngf * mult * 2, opt.mc), min(ngf, opt.mc), 1, 1)]
        if opt.feat_dim > 0:
            model += [nn.Conv2d(min(ngf * mult * 2, opt.mc), opt.feat_dim, 1, 1)]
        self.encoder = nn.Sequential(*model)

[Jak0917377107]

README.md