Hands-on Practical: Implementing Advanced MCMC

Advanced Markov Chain Monte Carlo (MCMC) methods, such as Hamiltonian Monte Carlo (HMC) and the No-U-Turn Sampler (NUTS), are often more efficient than simpler methods like Metropolis-Hastings or Gibbs sampling, especially for complex, high-dimensional posteriors. This efficiency stems from their underlying mechanics. Implementation and diagnosis of these samplers using a modern probabilistic programming library are covered.

We'll use Python with the popular library PyMC, which provides an intuitive interface for defining Bayesian models and utilizes powerful samplers (including NUTS) under the hood. We'll also use ArviZ for comprehensive MCMC diagnostics and visualization.

Setting Up Your Environment

First, ensure you have the necessary libraries installed. You can typically install them using pip:

pip install pymc arviz numpy pandas matplotlib seaborn

We'll use these libraries to define a model, run the NUTS sampler, and analyze the results.

Defining a Bayesian Model: A Hierarchical Example

Let's tackle a common problem where advanced MCMC shines: a hierarchical model. Imagine we want to model a relationship (e.g., linear regression) across several distinct groups, allowing the regression parameters to vary by group but assuming these group-specific parameters come from a common population distribution. This "partial pooling" approach often provides better estimates than fitting completely separate models for each group or pooling all data together.

The Radon dataset is a classic example in Bayesian statistics. It involves measuring indoor radon levels in houses across different counties in Minnesota. We might want to model log-radon levels as a function of whether the measurement was taken in a basement, allowing the intercept and slope to vary by county.

Here's how we might specify such a hierarchical linear model in PyMC:

import pymc as pm
import numpy as np
import pandas as pd
import arviz as az
import matplotlib.pyplot as plt

# Assume 'radon_data' is a pandas DataFrame with columns:
# 'log_radon': log of radon measurement
# 'floor': 0 for basement, 1 for first floor
# 'county_idx': integer index representing the county
# 'n_counties': total number of unique counties

# Load or simulate data (replace with actual data loading)
# Example placeholder structure:
n_counties = 5 # Example
n_obs_per_county = 20 # Example
np.random.seed(42)
county_idx = np.repeat(np.arange(n_counties), n_obs_per_county)
floor = np.random.randint(0, 2, size=n_counties * n_obs_per_county)
# Simulate some underlying county effects
true_county_intercept = np.random.normal(1.5, 0.5, size=n_counties)
true_county_slope = np.random.normal(-0.6, 0.2, size=n_counties)
log_radon = (true_county_intercept[county_idx] +
             true_county_slope[county_idx] * floor +
             np.random.normal(0, 0.3, size=n_counties * n_obs_per_county))

radon_data = pd.DataFrame({
    'log_radon': log_radon,
    'floor': floor,
    'county_idx': county_idx
})
n_counties = radon_data['county_idx'].nunique()
county_idx = radon_data['county_idx'].values
floor_measure = radon_data['floor'].values
log_radon_obs = radon_data['log_radon'].values

# Define the hierarchical model
with pm.Model() as hierarchical_model:
    # Hyperpriors for county intercepts
    mu_a = pm.Normal('mu_a', mu=0., sigma=1.)
    sigma_a = pm.HalfCauchy('sigma_a', beta=1.)

    # Hyperpriors for county slopes
    mu_b = pm.Normal('mu_b', mu=0., sigma=1.)
    sigma_b = pm.HalfCauchy('sigma_b', beta=1.)

    # County-specific intercepts (non-centered parameterization)
    a_offset = pm.Normal('a_offset', mu=0, sigma=1, shape=n_counties)
    a = pm.Deterministic('a', mu_a + a_offset * sigma_a)

    # County-specific slopes (non-centered parameterization)
    b_offset = pm.Normal('b_offset', mu=0, sigma=1, shape=n_counties)
    b = pm.Deterministic('b', mu_b + b_offset * sigma_b)

    # Model error term
    eps = pm.HalfCauchy('eps', beta=1.)

    # Expected value based on county and floor
    radon_est = a[county_idx] + b[county_idx] * floor_measure

    # Data likelihood
    radon_like = pm.Normal('radon_like', mu=radon_est, sigma=eps, observed=log_radon_obs)

Notice the structure:

• mu_a, sigma_a, mu_b, sigma_b are hyperpriors governing the distribution of county-level parameters.
• a and b represent the intercepts and slopes for each county. We use a non-centered parameterization (a_offset, b_offset) which often improves sampler efficiency in hierarchical models compared to directly sampling a and b from Normal(mu_a, sigma_a).
• eps is the observation noise standard deviation.
• radon_like defines the likelihood, connecting the model predictions (radon_est) to the observed data (log_radon_obs).

Running the NUTS Sampler

With the model defined, running the NUTS sampler in PyMC is straightforward. The pm.sample() function intelligently chooses NUTS for continuous models.

# Run the NUTS sampler
with hierarchical_model:
    idata = pm.sample(draws=2000, tune=1000, chains=4, cores=4, target_accept=0.9)
    # draws: Number of samples to keep per chain
    # tune: Number of initial 'burn-in' steps per chain to discard (tunes the sampler)
    # chains: Number of independent Markov chains to run
    # cores: Number of CPU cores to use (runs chains in parallel)
    # target_accept: The desired average acceptance probability for NUTS steps (higher values can sometimes help with difficult posteriors)

PyMC executes the sampling process, providing progress bars and potential warnings (e.g., about divergences or low effective sample sizes). The result, idata (InferenceData), is an object structured according to ArviZ conventions, containing the posterior samples, sampler statistics, observed data, and more.

Analyzing Sampler Performance and Convergence

Simply running the sampler isn't enough. We need to check if the chains have converged to the target distribution and if the sampling was efficient. ArviZ provides excellent tools for this.

Summary Statistics: $\hat{R}$ and ESS

The az.summary() function gives a table of important statistics for each parameter:

summary = az.summary(idata, var_names=['mu_a', 'sigma_a', 'mu_b', 'sigma_b', 'eps', 'a', 'b'])
print(summary)

Look closely at two columns:

• r_hat ($\hat{R}$): The Gelman-Rubin diagnostic. It compares the variance within chains to the variance between chains. Values close to 1.0 (e.g., < 1.01) suggest convergence. Values much larger than 1 indicate that the chains haven't mixed well and likely haven't converged to the same distribution.
• ess_bulk and ess_tail: Effective Sample Size (ESS). This estimates the number of independent samples equivalent to the correlated samples obtained by MCMC. Low ESS values (relative to the total number of post-warmup draws) indicate high autocorrelation and inefficient sampling for that parameter. Bulk ESS relates to the center of the distribution (e.g., for estimating the mean), while tail ESS relates to the tails (e.g., for estimating quantiles). Aim for ESS values in the hundreds or thousands for reliable estimates.

Visual Diagnostics: Trace Plots

Trace plots show the sampled value of a parameter over the iterations for each chain. They are invaluable for visually inspecting convergence and mixing.

az.plot_trace(idata, var_names=['mu_a', 'sigma_a', 'mu_b', 'sigma_b', 'eps'])
plt.tight_layout()
plt.show()

For each parameter plotted, you'll see two panels:

1. Left Panel (Trace): Shows the sampled values across iterations for all chains. Converged chains should look like fuzzy caterpillars mixing together, exploring the same range of values without long-term trends.
2. Right Panel (Posterior Density): Shows the estimated marginal posterior density distribution for the parameter, combining samples from all chains.

Here's an example visualization concept for a trace plot using fixed data for demonstration:

Example trace plot for parameter mu_a. The left side shows iteration values for two chains; the right shows the combined posterior density estimate.

Other Useful Plots

• az.plot_posterior: Directly plots the posterior distribution with credible intervals.
• az.plot_pair: Creates a matrix of pairwise scatter plots for selected parameters, useful for identifying correlations in the posterior.
• az.plot_energy: Specific to HMC/NUTS, plots energy transitions diagnostics helpful for identifying sampling difficulties.

Interpreting Results and Addressing Issues

Once satisfied with convergence, you can use the posterior samples in idata.posterior to:

• Calculate posterior means, medians, and credible intervals for parameters.
• Make predictions (using pm.sample_posterior_predictive).
• Compare models (covered later in the course).

Handling Divergences

NUTS is generally reliable, but you might encounter "divergences" or "divergent transitions". These are warnings indicating that the sampler encountered numerical difficulties, often in regions of high curvature or complex geometry in the posterior. Divergences mean the samples might be biased. Common strategies include:

• Increasing target_accept: Setting target_accept closer to 1 (e.g., 0.95, 0.99) forces smaller step sizes, potentially navigating tricky geometry better, but at the cost of slower sampling.
• Reparameterization: As we did with the non-centered parameterization for a and b, changing how the model is specified mathematically (without changing the underlying model) can drastically improve the posterior geometry for the sampler.
• Stronger Priors: Very weak priors can sometimes lead to regions of the parameter space that are difficult to explore. Slightly more informative priors might help constrain the sampler.
• Model Misspecification: Persistent divergences might indicate the model itself is a poor fit for the data or structurally problematic.

Conclusion

This practical exercise demonstrated how to implement an advanced MCMC method (NUTS) using PyMC for a non-trivial hierarchical Bayesian model. We covered defining the model, running the sampler, and, significantly, performing essential convergence and performance diagnostics using ArviZ. Remember that MCMC is not a black box; careful analysis of the output through summary statistics and visual plots is necessary to ensure the reliability of your posterior inferences. Mastering these tools allows you to confidently apply sophisticated Bayesian models to complex problems.