Properties of SHAP Values

Derived from Shapley values in cooperative game theory, SHAP (SHapley Additive exPlanations) values provide a principled way to attribute the output of a model prediction to its input features. What makes SHAP values a particularly compelling approach? Their strength lies in a set of desirable mathematical properties they uniquely satisfy among additive feature attribution methods. Understanding these properties helps clarify why SHAP provides consistent and reliable explanations.

There are three primary properties inherited from Shapley values that SHAP explanations adhere to:

1. Local Accuracy (or Additivity)

This property ensures that the explanation provided for a single prediction accurately reflects the model's output for that instance. Specifically, the sum of the SHAP values ($\phi_i$) for all input features ($M$) for a given prediction equals the difference between the model's prediction for that instance ($f(x)$) and the average prediction across the entire dataset or a background set ($E[f(X)]$, also called the base value or expected value).

Mathematically, this is expressed as:

$$\sum_{i=1}^{M} \phi_i = f(x) - E[f(X)]$$

• $f(x)$ is the prediction of the model for the input instance $x$.
• $E[f(X)]$ is the expected value of the model output over the background dataset (often the mean prediction for regression or mean probability for classification). This serves as the baseline prediction when no feature information is known.
• $\phi_i$ is the SHAP value for feature $i$, representing its contribution to pushing the prediction away from the base value.

Significance: Local accuracy guarantees that the feature contributions fully add up to explain the difference between the specific prediction and the baseline. It provides a complete accounting of the prediction at the local level. If a feature pushes the prediction higher, it gets a positive SHAP value; if it pushes it lower, it gets a negative SHAP value. The sum of these effects precisely matches the model's deviation from the average.

2. Missingness

The missingness property states that features that genuinely have no impact on the prediction should be assigned a SHAP value of zero. If a feature $x_i$ is 'missing' in the sense that its value does not influence the model's output for the specific instance $x$ (perhaps because it wasn't included in a subset of features being evaluated, or its value is such that the model ignores it), its attributed importance should be null.

Symbolically, if feature $i$ has no marginal contribution, then:

$$\phi_i = 0$$

Significance: This aligns with intuition. Features that don't contribute to the prediction outcome shouldn't receive any credit or blame in the explanation. This prevents the attribution method from assigning spurious importance to irrelevant inputs.

3. Consistency (or Monotonicity)

Consistency is arguably the most important property differentiating SHAP from some other methods. It states that if a model changes such that the marginal contribution of a feature increases or stays the same (regardless of other features present), the SHAP value assigned to that feature should also increase or stay the same. It should never decrease if the feature's impact becomes strictly larger.

Let's consider two models, $f$ and $f'$. If, for a specific feature $i$ and all possible subsets $S$ of the other features, the contribution of feature $i$ in model $f'$ is greater than or equal to its contribution in model $f$:

$$f'(x_{S \cup \{i\}}) - f'(x_S) \ge f(x_{S \cup \{i\}}) - f(x_S)$$

Then, the consistency property guarantees that the SHAP value for feature $i$ will also reflect this increased importance:

$$\phi_i(f', x) \ge \phi_i(f, x)$$

Significance: Consistency ensures that the feature importance measures align with how the model actually uses the features. If a feature becomes uniformly more important in the model logic, its SHAP value won't paradoxically decrease. This provides a guarantee that SHAP values are a faithful representation of feature impact, making the explanations more reliable and trustworthy compared to methods that might violate this property. For example, standard feature importance metrics derived from tree-based models (like gain or permutation importance) do not always satisfy consistency.

These three properties, rooted in game theory, provide SHAP with a solid theoretical underpinning. They ensure that the explanations are accurate at the local level (Local Accuracy), ignore irrelevant features (Missingness), and reliably reflect changes in feature contribution (Consistency). This foundation makes SHAP a powerful and widely adopted framework for interpreting complex machine learning models. Having established these properties, we can now look at practical methods for calculating SHAP values.