Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Batchnorm for GCN

See original GitHub issue

❓ Questions & Help

So I am not sure how I would implement a batchnorm layer if I am using a GCN. After a Convolution I would get a matrix of size [nodes_per_graph*batchsize, features]. But the nodes_per_graph differ between graphs so some batches haves more rows than others.

Now would I still perform a normilaization on each column, even if the size differs from batch to batch? Am I missing something obvious?

Issue Analytics

State:
Created 3 years ago
Comments:9 (3 by maintainers)

Top GitHub Comments

2reactions

JanoschMenkecommented, Oct 5, 2020

In this paper they show that Batchnorm can decrease the expressivness of GNN. Maybe their suggestion, GraphNorm, could be useful https://arxiv.org/pdf/2009.03294.pdf

1reaction

rusty1scommented, Oct 5, 2020

You can try using batch norm with track_running_stats=False.

Top Results From Across the Web

Batchnorm for GCN · Issue #1066 · pyg-team/pytorch_geometric

In general, when we're using batch normalization here, it normalizes features over all the nodes of the graph (including both test and train ......

A Principled Approach to Accelerating Graph Neural Network ...

normalization (BatchNorm) is a standard component in com- puter vision (Ioffe & Szegedy, 2015); Layer ... Graph Convolutional Networks (GCN) (Kipf &.

dgllife.model.gnn.gcn — DGL-LifeSci 0.2.9 documentation

``len(residual)`` equals the number of GCN layers. By default, residual connection is performed for each GCN layer. batchnorm : list of bool ......

LEARNING GRAPH NORMALIZATION FOR GRAPH NEURAL ...

It is well known that one of the critical ingredients to effectively train deep neural networks is normalization technique, e.g., Batch Normalization (BN)...

Part 2 – Comparing Message-Passing-Based GNN Architectures

The GatedGCN architecture is an anisotropic message-passing based GCN. It employs residual connections, batch normalization, and edge gates.