GCNII: a 64-layer GCN on Cora
Author: K3-Node Team
Backend: Multi-Backend
Dataset: Cora (Planetoid)
Description: Classify papers in the Cora citation network by topic with a very deep (64-layer) GCN.
GCNII: a 64-layer GCN on Cora
Classify papers in the Cora citation network by topic with a very deep (64-layer) GCN. GCNII (Chen et al., 2020) avoids over-smoothing by mixing the first layer's representation back in at every layer (initial residual) and keeping each layer close to the identity (identity mapping).
Same model as PyG's examples/gcn2_cora.py; trained for 200 epochs instead of PyG's 1000 to keep the notebook quick.
Install K3-Node, then choose a backend: "tensorflow", "torch" or "jax"
Load the data
data holds one graph: node features x, edges edge_index, labels y, and masks marking the training, validation and test nodes.
import keras
from keras import ops
from k3_node.datasets import Planetoid
from k3_node.loader import FullGraphDataset
from k3_node.transforms import NormalizeFeatures
dataset = Planetoid("data/Planetoid", name="Cora", transform=NormalizeFeatures())
data = dataset[0]
print(data)
Define the model
from k3_node.layers import GCN2Conv
class GCNII(keras.Model):
def __init__(self, in_channels, hidden_channels, out_channels, num_layers, alpha, theta, dropout):
super().__init__()
self.dropout = keras.layers.Dropout(dropout)
self.lin_in = keras.layers.Dense(hidden_channels, activation="relu")
self.convs = [GCN2Conv(hidden_channels, alpha, theta, layer + 1) for layer in range(num_layers)]
self.lin_out = keras.layers.Dense(out_channels)
def call(self, data, training=False):
x = self.dropout(data.x, training=training)
x = x_0 = self.lin_in(x)
for conv in self.convs:
x = self.dropout(x, training=training)
x = ops.relu(conv(x, x_0, data.edge_index))
x = self.dropout(x, training=training)
return self.lin_out(x)
model = GCNII(dataset.num_features, hidden_channels=64, out_channels=dataset.num_classes,
num_layers=64, alpha=0.1, theta=0.5, dropout=0.6)
Train
FullGraphDataset feeds the whole graph to Keras; mask selects which nodes count in the loss and the accuracy.
model.compile(
optimizer=keras.optimizers.Adam(learning_rate=0.01, weight_decay=0.0005),
loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
weighted_metrics=["accuracy"],
)
model.fit(
FullGraphDataset(data, mask="train_mask"),
validation_data=FullGraphDataset(data, mask="val_mask"),
epochs=200,
verbose=2,
)