Definitions
Graph Neural Network
A Graph Neural Network (GNN) is a neural network acting on a graph with features on nodes (and possibly on edges). Most GNNs are message passing networks: layer updates every node by
where are the neighbours of , and are learned functions shared by all nodes, and is a permutation-invariant aggregation such as a sum, mean or maximum. Hence the network is equivariant under relabelling the nodes, and after layers depends on the -hop neighbourhood of . Node outputs serve node-level tasks; pooling all gives graph-level outputs. A temporal GNN, the subject of this paper, handles graphs whose edges or features change in time, either as a sequence of snapshots or as a continuous-time stream of timestamped events, typically by combining message passing with a recurrent or memory component per node (see Recurrent Neural Network).
Convolution Neural Network
A Convolution Neural Network (CNN) is a neural network for data on a regular grid (images, audio, time series) whose main layers are discrete convolutions, in practice cross-correlations, , with a small learned kernel applied at every position. Weight sharing makes the layers equivariant under translations and uses far fewer parameters than a dense layer; pooling layers then add approximate translation invariance and enlarge the receptive field. A CNN is a GNN on the grid graph in which the aggregation keeps track of the relative position of each neighbour.
GAN
A generative adversarial network, GAN (Goodfellow et al. 2014, arXiv:1406.2661) consists of a generator maps noise to samples, and a discriminator outputs the probability that its input is real. They play the minimax game
For fixed the optimal discriminator is , and the value becomes , so the generator minimises the Jensen-Shannon divergence to the data distribution.
RAG
Retrieval-augmented generation, RAG (Lewis et al. 2020, arXiv:2005.11401) uses a language model whose output is conditioned on documents retrieved from an external corpus. A retriever embeds the query and returns the nearest documents in an index, usually by inner product of dense embeddings, and the generator conditions on the query together with the retrieved text. Knowledge then lives in an index that can be updated without retraining, and the retrieved passages can be cited.
Summary
The paper (Farzaneh, 2024) gives an information-theoretic framework for temporal GNNs. It adapts the information bottleneck principle, which seeks representations that keep the information relevant to the target while compressing the input, to the temporal setting, and introduces a definition of mutual information rate to study how much information such networks carry over time.