Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Problem in GCN link prediction example

See original GitHub issue

Describe the bug

It seems that the GCN link prediction example doesn’t perform the same performance in the doc. And the GCN seems didn’t learn something from the data, I am not sure why this happen.

To Reproduce

Steps to reproduce the behavior:

Go to the colab example code https://colab.research.google.com/github/stellargraph/stellargraph/blob/master/demos/link-prediction/gcn-link-prediction.ipynb#scrollTo=cW8U-jvPKdob.
Just run all the commands.
Can’t get the same performance in https://stellargraph.readthedocs.io/en/stable/demos/link-prediction/gcn-link-prediction.html, the acc is close to 0 in the reproduction of example.

I just run the code w/o any modification.

Environment

Operating system: 1. Ubuntu 2. colab

Python version: 1. python3.6.9 2. colab example default

Package versions: 1. stellargraph==1.2.1, tensorflow==2.2.0 2. colab example default

<IPython.core.display.HTML object>
StellarGraph: Undirected multigraph
 Nodes: 2708, Edges: 5429

 Node types:
  paper: [2708]
    Features: float32 vector, length 1440
    Edge types: paper-cites->paper

 Edge types:
    paper-cites->paper: [5429]
        Weights: all 1 (default)
        Features: none
** Sampled 542 positive and 542 negative edges. **
** Sampled 488 positive and 488 negative edges. **
Using GCN (local pooling) filters...
Using GCN (local pooling) filters...
2020-07-08 02:02:51.125100: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10
1/1 [==============================] - 0s 261us/step - loss: 1.7441 - acc: 0.0000e+00
1/1 [==============================] - 0s 372us/step - loss: 1.7190 - acc: 0.0000e+00

Train Set Metrics of the initial (untrained) model:
        loss: 1.7441
        acc: 0.0000

Test Set Metrics of the initial (untrained) model:
        loss: 1.7190
        acc: 0.0000
Epoch 1/50
1/1 - 0s - loss: 1.7230 - acc: 0.0000e+00 - val_loss: 1.6174 - val_acc: 0.0000e+00
Epoch 2/50
1/1 - 0s - loss: 1.8744 - acc: 0.0000e+00 - val_loss: 0.6929 - val_acc: 0.0000e+00
Epoch 3/50
1/1 - 0s - loss: 0.7559 - acc: 0.0000e+00 - val_loss: 0.8678 - val_acc: 0.0000e+00
Epoch 4/50
1/1 - 0s - loss: 0.8746 - acc: 0.0000e+00 - val_loss: 0.9703 - val_acc: 0.0000e+00
Epoch 5/50
1/1 - 0s - loss: 0.9689 - acc: 0.0000e+00 - val_loss: 0.9082 - val_acc: 0.0000e+00
Epoch 6/50
1/1 - 0s - loss: 0.8899 - acc: 0.0000e+00 - val_loss: 0.7689 - val_acc: 0.0000e+00
Epoch 7/50
1/1 - 0s - loss: 0.7362 - acc: 0.0000e+00 - val_loss: 0.6734 - val_acc: 0.0000e+00
Epoch 8/50
1/1 - 0s - loss: 0.6950 - acc: 0.0000e+00 - val_loss: 0.7262 - val_acc: 0.0000e+00
Epoch 9/50
1/1 - 0s - loss: 0.7597 - acc: 0.0000e+00 - val_loss: 0.9009 - val_acc: 0.0000e+00
Epoch 10/50
1/1 - 0s - loss: 1.0559 - acc: 0.0000e+00 - val_loss: 0.8252 - val_acc: 0.0000e+00
Epoch 11/50
1/1 - 0s - loss: 0.9358 - acc: 0.0000e+00 - val_loss: 0.6823 - val_acc: 0.0000e+00
Epoch 12/50
1/1 - 0s - loss: 0.7335 - acc: 0.0000e+00 - val_loss: 0.6319 - val_acc: 0.0000e+00
Epoch 13/50
1/1 - 0s - loss: 0.5645 - acc: 0.0000e+00 - val_loss: 0.6511 - val_acc: 0.0000e+00
Epoch 14/50
1/1 - 0s - loss: 0.6124 - acc: 0.0000e+00 - val_loss: 0.6859 - val_acc: 0.0000e+00
Epoch 15/50
1/1 - 0s - loss: 0.6148 - acc: 0.0000e+00 - val_loss: 0.7000 - val_acc: 0.0000e+00
Epoch 16/50
1/1 - 0s - loss: 0.6342 - acc: 0.0000e+00 - val_loss: 0.6994 - val_acc: 0.0000e+00
Epoch 17/50
1/1 - 0s - loss: 0.6002 - acc: 0.0000e+00 - val_loss: 0.6820 - val_acc: 0.0000e+00
Epoch 18/50
1/1 - 0s - loss: 0.5741 - acc: 0.0000e+00 - val_loss: 0.6816 - val_acc: 0.0000e+00
Epoch 19/50
1/1 - 0s - loss: 0.5270 - acc: 0.0000e+00 - val_loss: 0.6888 - val_acc: 0.0000e+00
Epoch 20/50
1/1 - 0s - loss: 0.5358 - acc: 0.0000e+00 - val_loss: 0.7120 - val_acc: 0.0000e+00
Epoch 21/50
1/1 - 0s - loss: 0.5582 - acc: 0.0000e+00 - val_loss: 0.7397 - val_acc: 0.0000e+00
Epoch 22/50
1/1 - 0s - loss: 0.6449 - acc: 0.0000e+00 - val_loss: 0.7800 - val_acc: 0.0000e+00
Epoch 23/50
1/1 - 0s - loss: 0.5558 - acc: 0.0000e+00 - val_loss: 0.6514 - val_acc: 0.0000e+00
Epoch 24/50
1/1 - 0s - loss: 0.5180 - acc: 0.0000e+00 - val_loss: 0.6666 - val_acc: 0.0000e+00
Epoch 25/50
1/1 - 0s - loss: 0.5070 - acc: 0.0000e+00 - val_loss: 0.6780 - val_acc: 0.0000e+00
Epoch 26/50
1/1 - 0s - loss: 0.5492 - acc: 0.0000e+00 - val_loss: 0.7167 - val_acc: 0.0000e+00
Epoch 27/50
1/1 - 0s - loss: 0.5637 - acc: 0.0000e+00 - val_loss: 0.7343 - val_acc: 0.0000e+00
Epoch 28/50
1/1 - 0s - loss: 0.6122 - acc: 0.0000e+00 - val_loss: 0.7493 - val_acc: 0.0000e+00
Epoch 29/50
1/1 - 0s - loss: 0.6122 - acc: 0.0000e+00 - val_loss: 0.7420 - val_acc: 0.0000e+00
Epoch 30/50
1/1 - 0s - loss: 0.5376 - acc: 0.0000e+00 - val_loss: 0.7160 - val_acc: 0.0000e+00
Epoch 31/50
1/1 - 0s - loss: 0.5866 - acc: 0.0000e+00 - val_loss: 0.6874 - val_acc: 0.0000e+00
Epoch 32/50
1/1 - 0s - loss: 0.5429 - acc: 0.0000e+00 - val_loss: 0.6786 - val_acc: 0.0000e+00
Epoch 33/50
1/1 - 0s - loss: 0.6190 - acc: 0.0000e+00 - val_loss: 0.6995 - val_acc: 0.0000e+00
Epoch 34/50
1/1 - 0s - loss: 0.5032 - acc: 0.0000e+00 - val_loss: 0.7124 - val_acc: 0.0000e+00
Epoch 35/50
1/1 - 0s - loss: 0.5199 - acc: 0.0000e+00 - val_loss: 0.7479 - val_acc: 0.0000e+00
Epoch 36/50
1/1 - 0s - loss: 0.5283 - acc: 0.0000e+00 - val_loss: 0.7146 - val_acc: 0.0000e+00
Epoch 37/50
1/1 - 0s - loss: 0.5227 - acc: 0.0000e+00 - val_loss: 0.6946 - val_acc: 0.0000e+00
Epoch 38/50
1/1 - 0s - loss: 0.4818 - acc: 0.0000e+00 - val_loss: 0.6404 - val_acc: 0.0000e+00
Epoch 39/50
1/1 - 0s - loss: 0.4635 - acc: 0.0000e+00 - val_loss: 0.6060 - val_acc: 0.0000e+00
Epoch 40/50
1/1 - 0s - loss: 0.4405 - acc: 0.0000e+00 - val_loss: 0.6063 - val_acc: 0.0000e+00
Epoch 41/50
1/1 - 0s - loss: 0.4395 - acc: 0.0000e+00 - val_loss: 0.6093 - val_acc: 0.0000e+00
Epoch 42/50
1/1 - 0s - loss: 0.4369 - acc: 0.0000e+00 - val_loss: 0.6352 - val_acc: 0.0000e+00
Epoch 43/50
1/1 - 0s - loss: 0.4165 - acc: 0.0000e+00 - val_loss: 0.6561 - val_acc: 0.0000e+00
Epoch 44/50
1/1 - 0s - loss: 0.3748 - acc: 0.0000e+00 - val_loss: 0.6558 - val_acc: 0.0000e+00
Epoch 45/50
1/1 - 0s - loss: 0.3929 - acc: 0.0000e+00 - val_loss: 0.6668 - val_acc: 0.0000e+00
Epoch 46/50
1/1 - 0s - loss: 0.3927 - acc: 0.0000e+00 - val_loss: 0.6897 - val_acc: 0.0000e+00
Epoch 47/50
1/1 - 0s - loss: 0.3800 - acc: 0.0000e+00 - val_loss: 0.7172 - val_acc: 0.0000e+00
Epoch 48/50
1/1 - 0s - loss: 0.3637 - acc: 0.0000e+00 - val_loss: 0.7382 - val_acc: 0.0000e+00
Epoch 49/50
1/1 - 0s - loss: 0.3720 - acc: 0.0000e+00 - val_loss: 0.7558 - val_acc: 0.0000e+00
Epoch 50/50
1/1 - 0s - loss: 0.3790 - acc: 0.0000e+00 - val_loss: 0.8127 - val_acc: 0.0000e+00
1/1 [==============================] - 0s 487us/step - loss: 0.3245 - acc: 0.0000e+00
1/1 [==============================] - 0s 266us/step - loss: 0.8127 - acc: 0.0000e+00

Train Set Metrics of the trained model:
        loss: 0.3245
        acc: 0.0000

Test Set Metrics of the trained model:
        loss: 0.8127
        acc: 0.0000

Issue Analytics

State:
Created 3 years ago
Comments:14 (8 by maintainers)

Top GitHub Comments

1reaction

davidho27941commented, Jul 10, 2020

@huonw

It’s ok. Thanks for helping me so much. BTW, I succeed to run GCN link pred. by stellargraph 1.2.1 with TF 2.0

Best, David

0reactions

huonwcommented, Jul 14, 2020

You’re pretty much correct, yeah.

I’ll rephrase the options, starting from the best/easiest:

replace acc with binary_accuracy: model.compile(..., metrics=["binary_accuracy"]) (which is what #1770 does in the demo in the repo)
remove the Reshapeing
use TF 2.1 with stellargraph 1.2.1 (that is, version 2.1 seems to work for me, not just 2.0), where fit, evaluate and predict should work too (no need to use ..._generator)

The last is much better than using TF 2.0, and I only noticed in my previous comment when working out the true underlying problem.

Top Results From Across the Web

Link prediction with GCN — StellarGraph 1.2.1 documentation

In this example, we use our implementation of the GCN algorithm to build a model that predicts citation links in the Cora dataset...

Graph Neural Networks: Link Prediction (Part II) | by Lina Faik

Task: Under this modeling, the problem becomes a link prediction task where the goal is to predict the label (rating) of a link...

Link Prediction Based on Graph Neural Networks

Link prediction is a key problem for network-structured data. Link prediction heuristics use some score functions, such as common neighbors and Katz index,....

SEAL Link Prediction, Explained - Towards Data Science

Graph Neural Networks (GNNs) have become very popular in recent years. You can do many things with them, like node label prediction, ...

Link Prediction using Graph Neural Networks - DGL Docs

This tutorial formulates the link prediction problem as a binary classification problem as follows: Treat the edges in the graph as positive examples....