Covariance Functions (Kernels): Properties and Selection

A Gaussian Process is completely specified by its mean function $m(\mathbf{x})$ and its covariance function $k(\mathbf{x}, \mathbf{x}')$. While the mean function defines the average expected value of the function at any point $\mathbf{x}$, it is the covariance function, often called the kernel, that truly defines the shape and structure of the functions that can be modeled. The kernel encodes assumptions about the function being learned, such as its smoothness, periodicity, or stationarity.

The Role of the Covariance Function (Kernel)

The kernel $k(\mathbf{x}, \mathbf{x}')$ quantifies the relationship between the function's values at two different input points, $\mathbf{x}$ and $\mathbf{x}'$. Specifically, it defines the covariance between the random variables $f(\mathbf{x})$ and $f(\mathbf{x}')$:

$$k(\mathbf{x}, \mathbf{x}') = \text{Cov}(f(\mathbf{x}), f(\mathbf{x}')) = \mathbb{E}[(f(\mathbf{x}) - m(\mathbf{x}))(f(\mathbf{x}') - m(\mathbf{x}'))]$$

If points $\mathbf{x}$ and $\mathbf{x}'$ are considered "similar" by the kernel (i.e., $k(\mathbf{x}, \mathbf{x}')$ is large), then we expect their corresponding output values $f(\mathbf{x})$ and $f(\mathbf{x}')$ to be close. Conversely, if the kernel deems them dissimilar ($k(\mathbf{x}, \mathbf{x}')$ is small or zero), their output values are less correlated.

For any finite set of $N$ input points $\mathbf{X} = \{\mathbf{x}_1, \dots, \mathbf{x}_N\}$, the kernel function defines the $N \times N$ covariance matrix $\mathbf{K}$, where each element $K_{ij} = k(\mathbf{x}_i, \mathbf{x}_j)$. This matrix is central to GP calculations.

Essential Properties of Kernels

For a function $k(\cdot, \cdot)$ to be a valid covariance function, it must satisfy two main properties:

1. Symmetry: The covariance between $f(\mathbf{x})$ and $f(\mathbf{x}')$ must be the same as between $f(\mathbf{x}')$ and $f(\mathbf{x})$. $$k(\mathbf{x}, \mathbf{x}') = k(\mathbf{x}', \mathbf{x})$$
2. Positive Semi-Definite (PSD): For any finite set of points $\{\mathbf{x}_1, \dots, \mathbf{x}_N\}$ and any real-valued vector $\mathbf{a} = [a_1, \dots, a_N]^T$, the resulting covariance matrix $\mathbf{K}$ must satisfy: $$\mathbf{a}^T \mathbf{K} \mathbf{a} \ge 0$$ This ensures that the variance of any linear combination of the function values, $\text{Var}(\sum_i a_i f(\mathbf{x}_i)) = \mathbf{a}^T \mathbf{K} \mathbf{a}$, is non-negative, a fundamental requirement for any valid covariance matrix. Mercer's theorem provides a deeper connection, stating that any continuous, symmetric, PSD kernel can be thought of as an inner product in some (potentially infinite-dimensional) feature space.

Interpreting Kernel Hyperparameters

Most useful kernels have parameters, often called hyperparameters, that control their behavior. Understanding these is significant for effective GP modeling. Two common hyperparameters are:

• Lengthscale ($l$): This parameter controls how quickly the correlation between function values decays as the distance between input points increases. A small lengthscale means correlation drops off quickly, leading to functions that vary rapidly (wiggly). A large lengthscale implies that points further apart remain correlated, resulting in smoother functions.
• Output Variance / Signal Variance ($\sigma_f^2$): This parameter scales the overall variance of the function values. It controls the typical amplitude or vertical range of the functions drawn from the GP prior. $k(\mathbf{x}, \mathbf{x}) = \sigma_f^2$ often represents the prior variance at any single point $\mathbf{x}$ (assuming a zero mean function).

The effect of the lengthscale is illustrated below, showing samples drawn from GPs with a Squared Exponential kernel but different lengthscales.

Function samples drawn from a zero-mean GP prior with a Squared Exponential kernel ($\sigma_f^2=1$). The purple line uses a shorter lengthscale ($l=0.5$), resulting in a rapidly varying function. The blue line uses a longer lengthscale ($l=2.0$), producing a much smoother function.

A Gallery of Common Kernels

Choosing the right kernel is often guided by prior knowledge about the function's expected characteristics. Here are some widely used kernels:

Squared Exponential (SE) Kernel / Radial Basis Function (RBF) Kernel

This is arguably the most common kernel, often a good starting point.

$$k_{SE}(\mathbf{x}, \mathbf{x}') = \sigma_f^2 \exp\left(-\frac{||\mathbf{x} - \mathbf{x}'||^2}{2l^2}\right)$$

• Properties: It produces very smooth functions (infinitely differentiable). It's stationary, meaning its value depends only on the displacement $||\mathbf{x} - \mathbf{x}'||$, not the absolute location of the points.
• Hyperparameters: Lengthscale $l$, output variance $\sigma_f^2$.
• ARD: For multi-dimensional inputs $\mathbf{x} \in \mathbb{R}^D$, you can use Automatic Relevance Determination (ARD) by assigning a separate lengthscale $l_d$ to each dimension: $$k_{SE-ARD}(\mathbf{x}, \mathbf{x}') = \sigma_f^2 \exp\left(-\frac{1}{2}\sum_{d=1}^D \frac{(x_d - x'_d)^2}{l_d^2}\right)$$ After optimization, dimensions with large $l_d$ values are effectively ignored, indicating low relevance for the prediction.

Matérn Kernels

The Matérn family offers more flexibility in controlling the smoothness of the function.

$$k_{Mat\acute{e}rn}(\mathbf{x}, \mathbf{x}') = \sigma_f^2 \frac{2^{1-\nu}}{\Gamma(\nu)} \left(\sqrt{2\nu}\frac{r}{l}\right)^\nu K_\nu\left(\sqrt{2\nu}\frac{r}{l}\right)$$

where $r = ||\mathbf{x} - \mathbf{x}'||$, $\Gamma(\cdot)$ is the gamma function, and $K_\nu(\cdot)$ is the modified Bessel function of the second kind.

• Properties: The parameter $\nu > 0$ controls the differentiability of the function samples. Functions drawn from a GP with a Matérn kernel are $k$-times differentiable if $\nu > k$. Common choices are $\nu = 3/2$ (once differentiable) and $\nu = 5/2$ (twice differentiable), which are often more realistic for physical processes than the infinite differentiability of the SE kernel. As $\nu \to \infty$, the Matérn kernel converges to the SE kernel. The case $\nu = 1/2$ yields the Exponential Kernel, which produces continuous but not differentiable functions (rough paths, similar to an Ornstein-Uhlenbeck process). Matérn kernels are stationary.
• Hyperparameters: Lengthscale $l$, output variance $\sigma_f^2$, smoothness parameter $\nu$. Often $\nu$ is fixed (e.g., to 3/2 or 5/2) rather than optimized.

Periodic Kernel

Ideal for modeling functions that exhibit repetitive patterns.

$$k_{Per}(\mathbf{x}, \mathbf{x}') = \sigma_f^2 \exp\left(-\frac{2 \sin^2(\pi ||\mathbf{x} - \mathbf{x}'|| / p)}{l^2}\right)$$

• Properties: Models functions that repeat with a period $p$. The lengthscale $l$ here controls the smoothness within a period. It's useful for seasonal data or signals with known frequencies. Stationary.
• Hyperparameters: Lengthscale $l$, output variance $\sigma_f^2$, period $p$.

Linear Kernel

Models simple linear relationships.

$$k_{Lin}(\mathbf{x}, \mathbf{x}') = \sigma_b^2 + \sigma_v^2 (\mathbf{x} - c)(\mathbf{x}' - c)^T$$

(Other variations exist, e.g., $k_{Lin}(\mathbf{x}, \mathbf{x}') = \sigma_b^2 + \sigma_v^2 \mathbf{x}^T \mathbf{x}'$).

• Properties: Generates functions that are linear trends. It's non-stationary because the covariance depends on the absolute locations $\mathbf{x}$ and $\mathbf{x}'$, not just their difference. The parameter $c$ represents an offset. $\sigma_b^2$ is a constant offset variance, and $\sigma_v^2$ controls the variance of the slope.
• Hyperparameters: Offset variance $\sigma_b^2$, slope variance $\sigma_v^2$, offset $c$.

Below is a comparison of samples from different kernel types.

Function samples drawn from zero-mean GP priors ($\sigma_f^2=1$) using different kernels. The Squared Exponential (SE) produces the smoothest sample. The Matérn 3/2 sample is less smooth (once differentiable). The Periodic sample clearly shows a repeating pattern.

Kernel Engineering: Combining Kernels

The real power of kernels often comes from combining basic kernels to capture more complex structures in the data. If $k_1$ and $k_2$ are valid kernels, then the following combinations are also valid kernels:

• Addition: $k_{sum}(\mathbf{x}, \mathbf{x}') = k_1(\mathbf{x}, \mathbf{x}') + k_2(\mathbf{x}, \mathbf{x}')$ This corresponds to modeling the function $f(\mathbf{x})$ as the sum of two independent Gaussian Processes, $f(\mathbf{x}) = f_1(\mathbf{x}) + f_2(\mathbf{x})$, where $f_1 \sim \mathcal{GP}(0, k_1)$ and $f_2 \sim \mathcal{GP}(0, k_2)$. This is extremely useful for modeling additive effects, like a long-term trend plus a seasonal component ($k_{Lin} + k_{Per}$).
• Multiplication: $k_{prod}(\mathbf{x}, \mathbf{x}') = k_1(\mathbf{x}, \mathbf{x}') \times k_2(\mathbf{x}, \mathbf{x}')$ Multiplication can create interesting interactions. For example, multiplying a smooth kernel (like SE) with a periodic kernel ($k_{SE} \times k_{Per}$) can model a periodic function whose amplitude changes smoothly over time. Multiplying kernels can also create non-stationary effects even if the base kernels are stationary.

By carefully combining kernels, you can encode sophisticated prior assumptions about the function's structure directly into the model.

Strategies for Kernel Selection

Choosing the appropriate kernel is a critical step in GP modeling. Here's a general approach:

1. Leverage Domain Knowledge: What do you know about the underlying process generating the data? Is it expected to be smooth (SE, Matérn)? Does it have cycles (Periodic)? Is there a baseline trend (Linear)? Start with kernels that reflect these assumptions.
2. Visual Inspection: Plotting the data can often reveal patterns like trends, seasonality, or abrupt changes that suggest suitable kernel structures or combinations.
3. Start Simple: Begin with standard kernels like SE or Matérn (e.g., Matérn 3/2 or 5/2). These are often flexible enough for many applications.
4. Consider Combinations: If simple kernels don't capture the structure, think about additive or multiplicative combinations (e.g., Trend + Seasonality, Smoothly varying amplitude).
5. Hyperparameter Optimization: Regardless of the initial choice, the kernel's hyperparameters ($l, \sigma_f^2, p$, etc.) must be tuned to the data. This is typically done by maximizing the marginal likelihood, which we will discuss in the next section.
6. Model Comparison: If you have several candidate kernels or kernel combinations, use model comparison techniques (like comparing marginal likelihoods or using cross-validation) to select the one that best explains the data.

The selection of the kernel fundamentally shapes the GP model. It defines the space of functions the GP considers plausible before seeing the data. A well-chosen kernel, combined with optimized hyperparameters, allows the GP to effectively learn from the data and make accurate predictions with meaningful uncertainty estimates. Next, we'll look into how to learn these important hyperparameters from data.