RuntimeError: Input type (torch.cuda.HalfTensor) and weight type (torch.cuda.FloatTensor) should be the same

I’ve encountered the following error when I’m trying to fine-tuning mel-spec from Tacotron2

Traceback (most recent call last):
  File "train.py", line 276, in <module>
    main()
  File "train.py", line 272, in main
    train(0, a, h)
  File "train.py", line 127, in train
    y_g_hat = generator(x)
  File "/home/kynh/anaconda3/envs/nguyenlm_hifigan/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "/home2/nguyenlm/Projects/hifi-gan-clone/models.py", line 101, in forward
    x = self.conv_pre(x)
  File "/home/kynh/anaconda3/envs/nguyenlm_hifigan/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/kynh/anaconda3/envs/nguyenlm_hifigan/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 202, in forward
    self.padding, self.dilation, self.groups)
RuntimeError: Input type (torch.cuda.HalfTensor) and weight type (torch.cuda.FloatTensor) should be the same

[leminhnguyen]

@Miralan Thanks you. After reading the code in meldataset.py file, I think there are some bugs in the code at the line audio = audio[:, mel_start * self.hop_size:(mel_start + frames_per_seg) * self.hop_size]. When I run the code I realize some the potential problems.

1. Assume the audio size is torch.Size([1, 36206]), frames_per_seg=32 and random mel_start=113. So we will slice audio[28928:37120]. It means the shape of audio after slicing is torch.Size([1, 7278] (because upper bound is 36206). That leads to RuntimeError: mismatch dimension. So as you said we can pad zeros
2. Assume the audio size is torch.Size([1, 72972]), frames_per_seg=32 and random mel_start=300. So we will slice audio[76800:84992]. It means the shape of audio after slicing is torch.Size([1, 0] (because we slice out of range of the audio). That leads to empty tensor. Again, we can overcome this problem by padding zeros, but I wonder if there are many empty tensors like that and filled out by zeros, it significantly affect to the quality of training or fine-tuning ?

[yulijun1220]

@leminhnguyen I’v encountered the same error. Do you have any suggestion on this? Traceback (most recent call last): File “train.py”, line 238, in <module> val_percent=args.val / 100) File “train.py”, line 128, in train_net loss.backward() File “C:\ProgramData\Anaconda3\envs\pytorch-py3.7\lib\site-packages\torch_tensor.py”, line 307, in backward torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs) File “C:\ProgramData\Anaconda3\envs\pytorch-py3.7\lib\site-packages\torch\autograd_init_.py”, line 156, in backward allow_unreachable=True, accumulate_grad=True) # allow_unreachable flag RuntimeError: Expected tensor for argument #1 ‘grad_output’ to have the same type as tensor for argument #2 ‘weight’; but type torch.cuda.HalfTensor does not equal torch.cuda.FloatTensor (while checking arguments for cudnn_convolution_backward_input) The source code link of train.py is https://github.com/milesial/Pytorch-UNet/blob/master/train.py