Multi-GPU VarNet training and batch size

My understanding is that the current VarNet training code uses batch size of 1 per GPU. Therefore in the multi-GPU training scenario the effective batch size would be num_gpus * batch_size = num_gpus as the gradients are averaged between GPUs after the backward pass.

According to the paper (and what I can also see in the code) the learning rate is set to 0.0003, but there is no mention of the (effective) batch size used in the experiment. Since the learning rate typically has to be adjusted to the batch size it would be good to know what batch size was used in the experiments (that is how many GPUs used). I expect changes in final SSIM on validation/test set with varying number of GPUs if the learning rate is kept at 0.0003.

[z-fabian]

VarNet only works with batch size 1, that is per GPU. This is because the slices have varying size. If you want to increase mini-batch size, that is number of training examples averaged per gradient update just increase the number of GPUs used for training.

[zhan4817]

However, when I tried to train VarNet with batch_size = 2, it shows the error in the validation sanity check:

… 237, in get_low_frequency_lines while mask[…, r, :]: RuntimeError: bool value of Tensor with more than one value is ambiguous

Do you have any idea? Thanks!