Implementing Membership Inference: Hands-on Practical

Implementing membership inference attacks involves determining if a specific data point was part of a target model's training set. This is often achieved by observing the model's behavior when processing that data point. Models typically respond differently, perhaps with higher confidence or specific output patterns, to data they have already seen during training compared to unseen data.

We'll focus on a common and effective approach: using shadow models. This technique involves training auxiliary models (the "shadow models") to mimic the behavior of the target model. We then train an "attack model" to distinguish between the outputs produced by these shadow models on their respective training (member) and testing (non-member) data. Finally, this attack model is used to classify outputs from the actual target model.

Setting the Stage

Imagine an attacker has query access to a trained target model, $f_{target}$ . The attacker also possesses some data similar in distribution to the target model's private training data, $D_{target\_train}$ . The attacker does not know $D_{target\_train}$ but wants to determine, for a given sample $x$ , whether $x \in D_{target\_train}$ .

The core steps for a shadow model-based MIA are:

Train Shadow Models: Create multiple models ( $f_{shadow\_1}, f_{shadow\_2}, ..., f_{shadow\_k}$ ) that ideally have the same architecture and are trained similarly to the target model $f_{target}$ . The attacker uses their own data, $D_{attacker}$ , partitioning it to train each shadow model. For each $f_{shadow\_i}$ , a portion of $D_{attacker}$ is used for its training set ( $D_{shadow\_i\_train}$ ), and another disjoint portion is kept aside as a test set ( $D_{shadow\_i\_test}$ ).
Generate Attack Training Data: Query each shadow model $f_{shadow\_i}$ with samples from its own training set ( $D_{shadow\_i\_train}$ ) and its test set ( $D_{shadow\_i\_test}$ ). Collect the outputs (e.g., prediction probability vectors or logits). Label the outputs derived from $D_{shadow\_i\_train}$ as "member" (label 1) and those from $D_{shadow\_i\_test}$ as "non-member" (label 0). Aggregate these labeled outputs from all shadow models to create a dataset for training the attack model.
Train the Attack Model: Train a binary classifier, $f_{attack}$ , using the dataset generated in the previous step. The features for this classifier are the output vectors (e.g., probabilities) from the shadow models, and the target variable is the member/non-member label.
Execute the Attack: Obtain the output of the target model $f_{target}$ for the sample $x$ you want to test. Feed this output vector into the trained attack model $f_{attack}$ . The prediction from $f_{attack}$ (either "member" or "non-member") is the result of the membership inference attack.

Implementation Outline

Let's illustrate this with pseudocode snippets, assuming you have functions define_model() which returns a model architecture (e.g., a Keras/PyTorch model) and train_model(model, train_data, epochs) which trains it. We'll also assume data is appropriately prepared (e.g., using TensorFlow Datasets or PyTorch DataLoaders).

1. Train Shadow Models

# Assume attacker_data is a dataset available to the attacker
# Assume num_shadow_models is the desired number of shadow models
# Assume target_model_architecture is known or approximated

shadow_model_outputs = []
shadow_model_labels = []

for i in range(num_shadow_models):
    print(f"Training shadow model {i+1}/{num_shadow_models}...")

    # Split attacker's data for this shadow model
    shadow_train_data, shadow_test_data = split_data(attacker_data, ratio=0.5) # Use disjoint partitions

    # Define and train the shadow model
    shadow_model = define_model(architecture=target_model_architecture)
    train_model(shadow_model, shadow_train_data, epochs=50) # Example epochs

    # Get predictions for its own training data (members)
    member_predictions = shadow_model.predict(shadow_train_data.x) # Get output vectors
    member_labels = np.ones(len(member_predictions))

    # Get predictions for its own test data (non-members)
    non_member_predictions = shadow_model.predict(shadow_test_data.x) # Get output vectors
    non_member_labels = np.zeros(len(non_member_predictions))

    # Store results for attack model training
    shadow_model_outputs.append(np.concatenate((member_predictions, non_member_predictions)))
    shadow_model_labels.append(np.concatenate((member_labels, non_member_labels)))

# Combine results from all shadow models
attack_train_X = np.concatenate(shadow_model_outputs)
attack_train_y = np.concatenate(shadow_model_labels)

print(f"Generated attack training data: {attack_train_X.shape}, {attack_train_y.shape}")

This loop simulates the process of creating proxies for the target model. Each shadow model learns patterns on its own data, and its behavior on seen vs. unseen data provides the signal for the attack model.

Flow of a membership inference attack using shadow models. The attacker trains shadow models on their own data to generate training data for an attack model. This attack model then predicts membership based on the output of the target model for a specific query data point.

2. Train the Attack Model

We can use a simple classifier like Logistic Regression or a small Multi-Layer Perceptron (MLP) from scikit-learn as the attack model.

from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score, classification_report
from sklearn.preprocessing import StandardScaler

# Optional: Scale the features (model outputs)
scaler = StandardScaler()
attack_train_X_scaled = scaler.fit_transform(attack_train_X)

# Split attack data for evaluation
X_train_att, X_test_att, y_train_att, y_test_att = train_test_split(
    attack_train_X_scaled, attack_train_y, test_size=0.3, stratify=attack_train_y, random_state=42
)

# Train the attack model
attack_model = LogisticRegression(solver='liblinear', random_state=42)
# Or use a small MLP:
# attack_model = MLPClassifier(hidden_layer_sizes=(64,), max_iter=300, random_state=42)

attack_model.fit(X_train_att, y_train_att)

# Evaluate on the held-out attack data split
y_pred_att = attack_model.predict(X_test_att)
print("Attack Model Performance (on shadow model data):")
print(f"Accuracy: {accuracy_score(y_test_att, y_pred_att):.4f}")
print(classification_report(y_test_att, y_pred_att))

This evaluation tells us how well the attack model learned to distinguish member vs. non-member outputs from the shadow models.

3. Execute the Attack on the Target Model

Now, the important step: use the trained attack_model on the target model's outputs. To properly evaluate the attack's success against the target, we need ground truth knowledge about the membership of some data points relative to $D_{target\_train}$ . This is usually possible in a research setting where we control the target model's training, but not in a real black-box scenario.

Let's assume we have target_model and access to its original training set target_train_data and test set target_test_data for evaluation purposes.

# Get target model outputs for known members (from its training set)
target_member_outputs = target_model.predict(target_train_data.x)

# Get target model outputs for known non-members (from its test set)
target_non_member_outputs = target_model.predict(target_test_data.x)

# Combine these outputs to form the test set for the attack model against the target
attack_test_target_X = np.concatenate((target_member_outputs, target_non_member_outputs))
attack_test_target_y = np.concatenate([np.ones(len(target_member_outputs)), np.zeros(len(target_non_member_outputs))])

# Scale using the same scaler fitted on shadow model outputs
attack_test_target_X_scaled = scaler.transform(attack_test_target_X) # Use transform, not fit_transform

# Predict membership using the attack model
final_predictions = attack_model.predict(attack_test_target_X_scaled)
final_probabilities = attack_model.predict_proba(attack_test_target_X_scaled)[:, 1] # Probability of being a member

# Evaluate the attack success on the target model
print("\nMembership Inference Attack Performance (on target model data):")
print(f"Accuracy: {accuracy_score(attack_test_target_y, final_predictions):.4f}")
print(classification_report(attack_test_target_y, final_predictions))

# You can also calculate AUC using final_probabilities and attack_test_target_y
# from sklearn.metrics import roc_auc_score
# print(f"AUC: {roc_auc_score(attack_test_target_y, final_probabilities):.4f}")

Evaluating Success and Interpretation

The main metric here is the final accuracy (or AUC) of the attack_model on the target model's data.

Accuracy close to 0.5 (50%): The attack is performing similarly to random guessing. This suggests the model leaks little membership information through its outputs, or the shadow models were not good proxies, or the attack model couldn't capture the distinguishing patterns.
Accuracy significantly higher than 0.5: The attack is successfully distinguishing members from non-members better than chance. This indicates a privacy leakage. The higher the accuracy, the more severe the leakage. An accuracy of 1.0 would mean perfect inference.

High precision for the "member" class means that when the attack predicts "member", it's likely correct. High recall means the attack identifies a large fraction of the true members.

Example evaluation results showing an attack performing noticeably better than random guessing, indicating potential membership information leakage from the target model.

Shadow Model Insights

Shadow Model Fidelity: The success of this attack heavily depends on how well the shadow models replicate the target model's properties (architecture, training data distribution, hyperparameters). Mismatches can degrade attack performance.
Output Type: Using full probability/confidence vectors often works better than just using the predicted label or the confidence of the predicted label. Logits can also be effective features for the attack model.
Data Requirements: The attacker needs sufficient data similar to the target's training data to train effective shadow models.
Defense Implications: Techniques like Differential Privacy (which we touched upon regarding its relationship to inference attacks) or output perturbation (adding noise to predictions) can make membership inference harder by obscuring the differences in model behavior between members and non-members. Regularization techniques during target model training might also inadvertently reduce leakage.

This hands-on perspective demonstrates that membership inference is not just a theoretical concern. With appropriate access and data, attackers can build models to probe the training history of machine learning systems, raising significant privacy questions. Understanding how to implement these attacks is the first step towards evaluating model vulnerabilities and designing effective defenses.

Was this section helpful?