Trouble to use Tensorboard's add_graph with my network

❓ Questions & Help

I am using code built upon this code example. Now I would like to add Tensorboard functionality.

Unfortunately, I am not able to get it work. I tried to adapt the insights from this tensorboard example provided at PyTorch Geometric’s repository.

Since the network expects a batch in the forward pass, I tried to use tensorboard.add_graph like this:

model = model.to(device)
for data in train_loader:
    data = data.to(device)
    model(data)
    break

writer.add_graph(model, data)

But then I get the following error message:

Type 'Tuple[Tuple[str, Tensor], Tuple[str, Tensor], Tuple[str, Tensor], Tuple[str, Tensor], Tuple[str, Tensor], Tuple[str, Tensor], Tuple[str, Tensor]]' cannot be traced. Only Tensors and (possibly nested) Lists, Dicts, and Tuples of Tensors can be traced (toTraceableStack at /opt/conda/conda-bld/pytorch_1579022030672/work/torch/csrc/jit/pybind_utils.h:305)

But the data I pass as a parameter to add_graph, next to the model should be some object, that can be forwarded through the network, shouldn’t it?

According to the tensorboard example I tried next:

model = model.to(device)
for data in train_loader:
    data = data.to(device)
    model(data)
    break

writer.add_graph(model, [data.x, data.edge_index])

But I get the following error message: TypeError: forward() takes 2 positional arguments but 3 were given

I really would like to now, how I can add my model to Tensorboard, since I would like to see, how the layers train and be able to do better debugging of my network.

Thank you in advance.

[rusty1s]

Tracing does not support arguments of arbitrary size. You should find the union set of arguments and pass them as optional parameters:

def forward(self, edge_index: Tensor, x: Optional[Tensor] = None, batch: Optional[Tensor] = None, ...):
    ...

[rusty1s]

Yes, this is a PyTorch jit limitation, since it cannot handle custom objects as arguments. You need to modify the forward call of model to: def forward(self, x, edge_index).