Running Error: Trying to backward through the graph a second time

🐛 Describe the bug

When I define a GAT net, and pass its output to BART model’s decoder, which uses for cross attention. The bart decoder has 12 attention layers. And I will get an Error: Trying to backward through the graph a second time. When I don‘t use GATConv layer, it won’t happen. Appreciate your help!!! The net is as below:

class GAT(torch.nn.Module):
    def __init__(self, in_channels, hidden_channels, out_channels, heads=2):
        super().__init__()
        self.conv1 = GATConv(in_channels, hidden_channels, heads=heads, dropout=0.6)
        self.conv2 = GATConv(heads*hidden_channels, out_channels, heads=1, concat=False, dropout=0.6)

    def forward(self, x, edge_index):
        x = F.dropout(x, p=0.2, training=self.training)
        x = self.conv1(x, edge_index)
        x = F.elu(x)
        x = F.dropout(x, p=0.2, training=self.training)
        x = self.conv2(x, edge_index)
        return F.log_softmax(x, dim=-1)

Environment

• PyG version: 1.7.2
• PyTorch version: 1.9.1
• OS: Linux
• Python version: 3.8
• CUDA/cuDNN version: 11.1

[rusty1s]

Do you have a minimal example to reproduce this? This would be helpful to track down the issue. Your GAT definition looks fine though.

[rusty1s]

Thanks for confirming. Hope that you can resolve your issue 😃 Eventually someone in the PyTorch forums can help you further.