On the learning stability of the results of the general_gnn.py

Hi Daniele and Jack,

I have a question regarding this example. In particular, running the code yields the following file’s content: data.txt.

However, if one plots the data above, say,

import matplotlib.pyplot as plt
import numpy as np
import matplotlib as mlp
from numpy import minimum, maximum

test_acc, epoch = [], []
for line in open('data.txt', 'r'):
  values = [s for s in line.split()]
  epoch.append(values[1]) 
  test_acc.append(float(values[15]))

plt.figure(figsize=(20,5))
plt.plot(epoch, test_acc)
plt.xlabel('Epoch')
plt.ylabel('Test accuracy')
plt.legend(["test_acc"], loc ="lower right")

plt.show()

there is no convergence and stability in terms of the test accuracy:

Here test_acc severely oscillates. Can you please explain how this learning process is considered to be normal while the accuracy of the model does not overall increase from one epoch to another?

[danielegrattarola]

The isssue here is that “molhiv” is a dataset that has edge attributes, but GeneralGNN expects only node attributes (x, a, i).

You can either change dataset or implement a model similar to GeneralGNN which is designed to discard edge attributes. Something like:

class MyGeneralGNN(GeneralGNN):
    def call(self, inputs):
        x, a, e, i = inputs
        return super().call([x,a, i])

[Matin-Macktoobian]

Thanks. So, I ended up with the following code:

import numpy as np
import matplotlib.pyplot as plt

from tensorflow.keras.losses import CategoricalCrossentropy
from tensorflow.keras.metrics import categorical_accuracy
from tensorflow.keras.optimizers import Adam

from spektral.data import DisjointLoader
from spektral.models import GeneralGNN

from spektral.datasets.ogb import OGB
from ogb.graphproppred import GraphPropPredDataset

ogb_dataset = GraphPropPredDataset(name="ogbg-molhiv")
dataset = OGB(ogb_dataset)

idx = ogb_dataset.get_idx_split()
idx_tr, idx_va, idx_te = idx["train"], idx["valid"], idx["test"]

dataset_tr = dataset[idx_tr]
dataset_va = dataset[idx_va]
dataset_te = dataset[idx_te]

np.random.seed(0)

batch_size = 16
learning_rate = 0.0001
epochs = 100

loader_tr = DisjointLoader(dataset_tr, batch_size=batch_size, epochs=epochs)
loader_te = DisjointLoader(dataset_te, batch_size=batch_size, epochs=1)

model = GeneralGNN(dataset.n_labels, activation="softmax")

optimizer = Adam(learning_rate)
loss_fn = CategoricalCrossentropy()
model.compile(loss=loss_fn,
              optimizer=optimizer,
              metrics=categorical_accuracy)

history = model.fit(loader_tr.load(), steps_per_epoch=loader_te.steps_per_epoch, epochs=epochs)

and I get the following error:

Traceback (most recent call last):
  File "~/PycharmProjects/RL/test_OGB_spektral.py", line 80, in <module>
    history = model.fit(loader_tr.load(), steps_per_epoch=loader_te.steps_per_epoch, epochs=epochs)
  File "~\AppData\Roaming\Python\Python37\site-packages\tensorflow\python\keras\engine\training.py", line 1183, in fit
    tmp_logs = self.train_function(iterator)
  File "~\AppData\Roaming\Python\Python37\site-packages\tensorflow\python\eager\def_function.py", line 889, in __call__
    result = self._call(*args, **kwds)
  File "~\AppData\Roaming\Python\Python37\site-packages\tensorflow\python\eager\def_function.py", line 933, in _call
    self._initialize(args, kwds, add_initializers_to=initializers)
  File "~\AppData\Roaming\Python\Python37\site-packages\tensorflow\python\eager\def_function.py", line 764, in _initialize
    *args, **kwds))
  File "~\AppData\Roaming\Python\Python37\site-packages\tensorflow\python\eager\function.py", line 3050, in _get_concrete_function_internal_garbage_collected
    graph_function, _ = self._maybe_define_function(args, kwargs)
  File "~\AppData\Roaming\Python\Python37\site-packages\tensorflow\python\eager\function.py", line 3444, in _maybe_define_function
    graph_function = self._create_graph_function(args, kwargs)
  File "~\AppData\Roaming\Python\Python37\site-packages\tensorflow\python\eager\function.py", line 3289, in _create_graph_function
    capture_by_value=self._capture_by_value),
  File "~\AppData\Roaming\Python\Python37\site-packages\tensorflow\python\framework\func_graph.py", line 999, in func_graph_from_py_func
    func_outputs = python_func(*func_args, **func_kwargs)
  File "~\AppData\Roaming\Python\Python37\site-packages\tensorflow\python\eager\def_function.py", line 672, in wrapped_fn
    out = weak_wrapped_fn().__wrapped__(*args, **kwds)
  File "~\AppData\Roaming\Python\Python37\site-packages\tensorflow\python\framework\func_graph.py", line 986, in wrapper
    raise e.ag_error_metadata.to_exception(e)
ValueError: in user code:

    ~\AppData\Roaming\Python\Python37\site-packages\tensorflow\python\keras\engine\training.py:855 train_function  *
        return step_function(self, iterator)
    ~\AppData\Local\Programs\Python\Python37\lib\site-packages\spektral\models\general_gnn.py:166 call  *
        x, a, i = inputs

    ValueError: too many values to unpack (expected 3)

The problem here is that in the examples you supplied, the models are created based on three values dataset.n_node_features, dataset.n_edge_features and dataset.n_labels . However, I can only dataset.n_labels as the output to the GeneralGNN constructor. If that’s the case, can you please explain how I can feed those values to GeneralGNN?