The previous section introduced Shapley values from cooperative game theory, a method for fairly distributing a "payout" among cooperating players. The SHAP (SHapley Additive exPlanations) framework cleverly adapts this concept to the context of machine learning model explanations.

Imagine trying to explain a single prediction made by your trained model for a specific input instance. How much did each feature of that instance contribute to the final prediction value, compared to some baseline? SHAP answers this by framing the problem as a cooperative game:

The Game: The process of generating the prediction for one specific instance, $x$ .
The Players: The individual features (e.g., age, income, zip code) of the instance $x$ .
The Payout: The difference between the model's actual prediction for instance $x$ , denoted as $f(x)$ , and the average prediction across the entire training dataset (or a representative background dataset), denoted as $E[f(X)]$ . This difference, $f(x) - E[f(X)]$ , represents the total contribution of all features specific to instance $x$ that moved the prediction away from the average.

The goal is to fairly distribute this total payout ( $f(x) - E[f(X)]$ ) among the "players" (the features). A feature's contribution, its SHAP value, represents how much that specific feature's value, in the context of other feature values for that instance, shifted the prediction.

Formally, the SHAP value $\phi_i(x)$ for feature $i$ and instance $x$ is calculated as the weighted average of its marginal contribution across all possible subsets (coalitions) of features not including feature $i$ . The "marginal contribution" of feature $i$ to a specific coalition $S$ (where $S$ is a subset of features excluding $i$ ) is the difference in the model's expected output when feature $i$ is added to the coalition versus the expected output with the coalition $S$ alone:

$E[f(X) | X_S \cup \{x_i\}] - E[f(X) | X_S]$

Here, $E[f(X) | X_S]$ represents the expected prediction given only the feature values for the features in subset $S$ . Calculating this conditional expectation is often the most complex part and requires making assumptions about feature independence or using specific calculation techniques, which we'll cover when discussing KernelSHAP and TreeSHAP.

SHAP values inherit a highly desirable property from Shapley values called additivity. This means that the sum of the SHAP values for all features of a given instance $x$ equals the difference between the prediction for that instance $f(x)$ and the base value (the average prediction $E[f(X)]$ ). We can express this relationship as:

f(x) = \phi_0 + \sum_{i=1}^{M} \phi_i(x)

where:

$f(x)$ is the model's prediction for instance $x$ .
$\phi_0 = E[f(X)]$ is the base value, representing the average model prediction over the background dataset.
$\phi_i(x)$ is the SHAP value for feature $i$ for the specific instance $x$ .
$M$ is the total number of features.

This equation provides a clear decomposition of the prediction. $\phi_0$ gives us the starting point (the average prediction), and each $\phi_i(x)$ tells us how the value of feature $i$ pushes the prediction higher (if $\phi_i(x)$ is positive) or lower (if $\phi_i(x)$ is negative) relative to this base value. The magnitude $|\phi_i(x)|$ indicates the strength of the feature's influence on this specific prediction.

Unlike LIME, which builds a local, interpretable surrogate model, SHAP directly calculates feature contributions based on solid game-theoretic principles, aiming for a more theoretically grounded attribution. However, computing these exact contributions across all possible feature coalitions is computationally infeasible for all but the simplest models and smallest feature sets. This computational challenge motivates the development of efficient approximation algorithms like KernelSHAP and TreeSHAP, which we will explore next.