Understanding Graph Convolutional Networks through Heat Diffusion and Laplacian Operators

Heat Diffusion Model on Graphs

Starting from the classic one‑dimensional heat conduction problem, the article discretises a uniform metal rod into a chain of nodes and derives the discrete heat equation, highlighting that the second‑order spatial derivative becomes a difference‑of‑differences term.

In Euclidean space the temperature change at a point is proportional to the Laplacian of the temperature field; the Laplacian is the sum of second‑order derivatives, which in the discrete case corresponds to the graph Laplacian matrix.

Extending the Model to Graphs

When the same diffusion process is placed on an arbitrary graph, each node exchanges heat only with its adjacent nodes. Using Newton’s cooling law, the temperature dynamics can be written as d\mathbf{x}/dt = -L\mathbf{x} , where L is the graph Laplacian constructed from the adjacency matrix and the degree matrix.

The derivation shows that the Laplacian encodes exactly the influence of neighbouring nodes, and that the diffusion equation on a graph has the same mathematical form as the continuous heat equation.

Message Passing Interpretation

Replacing heat with generic features or messages, the same update rule becomes the core of Graph Convolutional Networks (GCNs): each node updates its representation by aggregating the differences with its neighbours, i.e., a message‑passing step.

This perspective reveals that convolution, Fourier transform, and spectral methods are merely different ways of solving the same underlying diffusion equation.

Numerical Solvability in Machine Learning

Discretising time turns the differential equation into an iterative update \mathbf{x}^{(t+1)} = (I - \alpha L)\mathbf{x}^{(t)} , which is exactly the propagation rule used in many GNN layers. The iteration can be performed on mini‑batches, and low‑order neighbourhoods can be used by truncating the spectral decomposition.

Properties of the Laplacian

The (unnormalised) Laplacian L = D - A is symmetric, positive‑semi‑definite, and its eigenvectors form an orthogonal basis for the graph signal space. Normalised variants correspond to different normalisation strategies of neighbour influence.

Analogy to Semi‑Supervised Learning

In a semi‑supervised setting, labelled nodes act as constant‑temperature heat sources. Unlabelled nodes are the isolated system that receives feature information from these sources until a new equilibrium is reached, which explains why GCNs naturally propagate label information across the graph.

Connection to GraphSAGE

GraphSAGE implements the same diffusion principle with a learnable aggregator: neighbours are summed (or pooled) and concatenated with the node’s own representation before a non‑linear transformation. This is a non‑linear extension of the linear heat‑diffusion update.

Conclusion

Understanding the Laplacian‑based heat equation provides a unified physical intuition for GCNs, message passing, and spectral graph theory, and clarifies why various GNN architectures (e.g., GraphSAGE) can be seen as different approximations of the same diffusion process.