MessagePassing.propagate() computes the contraction with the transpose

Short: if (edge_index, norm) represent the matrix M in the sense that M = to_scipy_sparse_matrix(edge_index, norm) then MessagePassing.propagate(edge_index, x, norm) computes M.T @ x, so M is implicitly transposed in the process

Example

# Create matrix M and vector x
row = np.array([0, 0])
col = np.array([1, 2])
norm = np.array([1, 1])
num_nodes = 3
x = np.array([[2, 3, 4], [5, 6, 7]]).T

# Compute M@x and M.T@x explicitly
M = scipy.sparse.coo_matrix((norm, (row, col)), shape=(num_nodes, num_nodes))
print("M:\n", M.toarray())
print("x:\n:", x)
print("M @ x:\n", M @ x)
print("M.T @ x:\n", M.T @ x)

# Compute MessagePassing.propagate
conv = torch_geometric.nn.MessagePassing()  # flow='source_to_target'
edge_index = torch.tensor([row, col])
conv_result = conv.propagate(edge_index=edge_index,
                             x=torch.tensor(x),
                             norm=norm)
print("conv:")
print(conv_result)

output:

M:
 [[0 1 1]
 [0 0 0]
 [0 0 0]]
x:
: [[2 5]
 [3 6]
 [4 7]]
M @ x:
 [[ 7 13]
 [ 0  0]
 [ 0  0]]
M.T @ x:
 [[0 0]
 [2 5]
 [2 5]]
conv:
tensor([[0, 0],
        [2, 5],
        [2, 5]])

So we see that M.T @ x is computed.

Comments: This is probably the way it is supposed to work, but at least for me it was not very intuitive, that’s why I wanted to mention it. If I understand correctly, the reason why it works that way is that in PyG edge_index=(row, col) = (source, target), while for M @ x one should have row=target and col=source.

While the transposition of M can be called a convention, and wouldn’t matter for symmetric M, this might be a source of confusion for directed graphs.

[rusty1s]

That is correct. One can change this by setting flow=target_to_source. I think this is very intuitive if you think about the graph connectivity as a set of (source, target) tuples. A lot of papers also define A_ji = 1 if there is an edge from i to j, which is IMO much more confusing. I tried my best to make this clear in the message passing documentation.

[rusty1s]

Sure.