While LIME provides valuable local insights by approximating models with simpler surrogates, it has certain limitations, particularly regarding the consistency of its explanations. We now turn to a different approach rooted in cooperative game theory: SHapley Additive exPlanations (SHAP). The foundation of SHAP lies in the concept of Shapley values, which offer a principled way to fairly distribute the "payout" of a cooperative game among its players.

The Cooperative Game Analogy

Imagine a group of people collaborating on a project where the combined effort produces a certain value or outcome. How can we fairly determine how much each individual contributed to the final result? This is the central question addressed by cooperative game theory, and Lloyd Shapley developed a solution in the 1950s, now known as Shapley values.

The core idea is to consider the marginal contribution of each player (person) when added to different possible subgroups (coalitions) of the other players. A player's Shapley value is their average marginal contribution across all possible orderings or sequences in which players could join the coalition.

Let's consider a simple example. Suppose three friends, Alice (A), Bob (B), and Charlie (C), work together on a small consulting project.

Working alone, A generates $10k, B generates$ 15k, C generates $8k.
Working in pairs: {A, B} generate $30k, {A, C} generate$ 25k, {B, C} generate $35k.
All three working together {A, B, C} generate $50k.

How much of the final $50k should be attributed to each person? Shapley values provide a method. We look at all possible orderings (permutations) in which they could have joined the "grand coalition" {A, B, C} and calculate each person's marginal contribution in each ordering.

Ordering (A, B, C):
- A joins first: Contribution = $10k$ (value of {A} - value of {})
- B joins next: Contribution = $20k$ (value of {A, B} - value of {A} = $30k -$ 10k)
- C joins last: Contribution = $20k$ (value of {A, B, C} - value of {A, B} = $50k -$ 30k)
Ordering (A, C, B):
- A joins first: Contribution = $10k$
- C joins next: Contribution = $15k$ (value of {A, C} - value of {A} = $25k -$ 10k)
- B joins last: Contribution = $25k$ (value of {A, C, B} - value of {A, C} = $50k -$ 25k)

We would repeat this for all $3! = 6$ possible orderings: (A, B, C), (A, C, B), (B, A, C), (B, C, A), (C, A, B), (C, B, A).

The Shapley value for a player is the average of their marginal contributions across all these orderings. Let's calculate it for Alice (A):

Ordering	A's Marginal Contribution
(A, B, C)	$10k$
(A, C, B)	$10k$
(B, A, C)	$15k$ ( $30k - 15k$ )
(B, C, A)	$15k$ ( $50k - 35k$ )
(C, A, B)	$17k$ ( $25k - 8k$ )
(C, B, A)	$15k$ ( $50k - 35k$ )
Average	$13.67k$

Similarly, we can calculate the Shapley values for Bob and Charlie. This method guarantees a fair distribution based on average marginal impact.

Connecting Shapley Values to Model Predictions

How does this relate to explaining machine learning models? The SHAP framework ingeniously adapts this concept:

The Game: The "game" is the prediction task for a single instance (e.g., predicting the probability of loan default for a specific applicant).
The Players: The "players" are the feature values for that instance (e.g., the applicant's income, credit score, age, loan amount).
The Payout: The "payout" is the difference between the model's prediction for this specific instance and the baseline or average prediction across the entire dataset (or a representative background dataset). This difference represents the total contribution of all features for this specific prediction, moving it away from the average.
The Shapley Value: The Shapley value for a specific feature (e.g., 'income') for a given prediction represents that feature's contribution to pushing the prediction away from the baseline value. It's the fair attribution of the prediction difference to each feature.

For example, if the average predicted probability of default is 0.10, and for a specific applicant, the model predicts 0.75, the total "payout" to be distributed is $0.75 - 0.10 = 0.65$ . SHAP aims to calculate how much each feature (income, credit score, etc.) contributed to this $0.65$ difference. A positive Shapley value for a feature means it pushed the prediction higher (towards default, in this case), while a negative value means it pushed the prediction lower.

The Power of Shapley Values

Why use this game theory approach? Shapley values have several desirable mathematical properties (which we'll discuss in the "Properties of SHAP Values" section) that make them theoretically sound for feature attribution:

Efficiency (Local Accuracy): The sum of the Shapley values for all features for a given instance equals the difference between the prediction for that instance and the baseline prediction. The contributions perfectly add up.
Symmetry: If two features contribute identically to all possible coalitions, they receive the same Shapley value.
Dummy: A feature that never changes the prediction outcome (regardless of the coalition it's added to) receives a Shapley value of zero.
Additivity: For ensemble models where predictions are combined (like averages or sums), the Shapley values can also be combined consistently.

Calculating exact Shapley values requires evaluating the model's output for every possible subset (coalition) of features. For models with many features ( $M$ ), this involves $2^M$ evaluations, which quickly becomes computationally intractable. This is where the practical SHAP algorithms, such as KernelSHAP and TreeSHAP, come into play. They provide efficient ways to estimate these theoretically grounded Shapley values, making this powerful concept applicable to real-world machine learning problems.

In the following sections, we will examine how SHAP values are calculated and interpreted using these practical algorithms.