Common Aggregation Functions

The AGGREGATE function is a primary operation within the message passing mechanism used in Graph Neural Networks. Its purpose involves taking a set of feature vectors from a node's neighbors, which can vary in size, and condensing them into a single, representative vector. This operation is essential because while a node in a graph can have many neighbors, the UPDATE step, which processes the aggregated information, requires a fixed-size input.

The aggregator must be a permutation-invariant function, meaning the order in which neighbor features are processed does not affect the output. This property is essential for graphs, as there is no natural ordering to a node's neighbors. Let's examine the most common aggregation functions that satisfy this requirement.

Mean Aggregation

The mean aggregator computes the element-wise average of the neighbor feature vectors. It provides a summary of the neighborhood's features, normalized by the number of neighbors.

Mathematically, the aggregated message $\mathbf{m}_{\mathcal{N}(v)}^{(l)}$ for a target node $v$ is calculated as:

\mathbf{m}_{\mathcal{N}(v)}^{(l)} = \frac{1}{|\mathcal{N}(v)|} \sum_{u \in \mathcal{N}(v)} \mathbf{h}_u^{(l)}

Here, $|\mathcal{N}(v)|$ is the degree (number of neighbors) of node $v$ , and $\mathbf{h}_u^{(l)}$ is the feature vector of a neighbor node $u$ at layer $l$ .

The primary characteristic of mean aggregation is its smoothing effect. By averaging, it ensures that the magnitude of the aggregated vector does not grow with the number of neighbors. This makes it a stable and often effective default choice. However, this same property can be a drawback. If one neighbor has a particularly informative feature, its signal may be diluted by the average, especially in dense neighborhoods.

Sum Aggregation

The sum aggregator simply adds the feature vectors of all neighbors together. This is perhaps the most direct way to combine information from the neighborhood.

The formula for sum aggregation is:

\mathbf{m}_{\mathcal{N}(v)}^{(l)} = \sum_{u \in \mathcal{N}(v)} \mathbf{h}_u^{(l)}

Unlike the mean, the sum is sensitive to the size of the neighborhood. The resulting aggregated vector's magnitude will scale with the node's degree. This can be useful if the number of connections is an important structural property you want the model to learn. For example, in a social network, a user with many friends might be represented differently than one with few. The main risk with sum aggregation is that for nodes with very high degrees, the summed values can become extremely large, potentially leading to numerical instability during training.

Max Aggregation

Max aggregation, also known as max pooling, takes the element-wise maximum across all neighbor feature vectors. Instead of summarizing the entire neighborhood, it identifies the most prominent feature values present among the neighbors.

The operation is defined as:

\mathbf{m}_{\mathcal{N}(v)}^{(l)} = \max_{u \in \mathcal{N}(v)} \{ \mathbf{h}_u^{(l)} \}

This operation is performed independently for each dimension of the feature vectors. Max pooling is effective at capturing representative or salient features. For instance, if each feature dimension represents the presence of a certain attribute, max aggregation identifies the strongest presence of that attribute within the neighborhood. It is less susceptible to the diluting effects of averaging and is not as sensitive to the number of neighbors as the sum aggregator.

Comparing Aggregation Functions

To better understand the differences between these functions, consider a target node $v$ with three neighbors, each having a two-dimensional feature vector.

A comparison of Mean, Sum, and Max aggregation for a target node v. Each aggregator produces a different summary of the neighbor feature vectors [1, 5], [4, 2], and [2, 2].

As the diagram shows:

Mean produces an "average" representation: [2.33, 3.00].
Sum produces a vector with a larger magnitude that reflects the total contribution: [7, 9].
Max picks out the most salient feature in each dimension: [4, 5], taking 4 from u2 and 5 from u1.

Choosing an Aggregator

The choice of aggregation function depends on the specific task and the nature of the graph data.

Mean is a safe, general-purpose starting point that works well in many scenarios.
Sum can be a more expressive alternative if node degrees are meaningful, though it may require normalization to maintain training stability.
Max is useful when the task involves identifying the single most influential neighbor or feature.

Some advanced GNN architectures, like GraphSAGE which we will see in the next chapter, treat the aggregator as a configurable hyperparameter, allowing you to experiment and find the one that works best for your problem. The next section will cover the UPDATE function, which takes the aggregated message and combines it with the node's own representation to produce its new state.

Was this section helpful?

References

Inductive Representation Learning on Large Graphs, William L. Hamilton, Rex Ying, Jure Leskovec, 2017 Advances in Neural Information Processing Systems, Vol. 30 (Curran Associates, Inc.) DOI: 10.55985/05f23d4e68e4d262 - Introduced GraphSAGE, formalizing the concept of learnable neighborhood aggregation functions, including mean, pooling (max), and LSTM aggregators.
Semi-Supervised Classification with Graph Convolutional Networks, Thomas N. Kipf, Max Welling, 2017 International Conference on Learning Representations (ICLR) DOI: 10.48550/arXiv.1609.02907 - This foundational work introduced Graph Convolutional Networks (GCNs), which utilize a specific form of normalized aggregation of neighbor features in their message passing scheme.
Graph Neural Networks: A Review of Methods and Applications, Jie Zhou, Ganqu Cui, Shengding Hu, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changcheng Li, Maosong Sun, 2020 AI Open, Vol. 1 DOI: 10.1016/j.aiopen.2021.01.001 - Provides an overview of Graph Neural Networks, discussing different message passing paradigms and common aggregation functions.
CS224W: Machine Learning with Graphs (Course Materials), Jure Leskovec and Stanford University, 2024 - Official course materials from a leading university program, explaining fundamental concepts of graph neural networks, including aggregation functions.