Michael Brenndoerfer | Data & AI, Private Equity, Technology

Michael Brenndoerfer

LIME (Local Interpretable Model-agnostic Explanations) › Example

Michael BrenndoerferJuly 27, 20253 min read

Machine Learning from ScratchAll chapters

Part 6 of 9Explainability

Chapter 52LIME (Local Interpretable Model-agnostic Explanations) › Example

Reading section 5 of 9 · Example · 3 minSection 5 of 9 · 3 min

Section 5 of 9 · 3 min read

LIME (Local Interpretable Model-agnostic Explanations) › ExampleOne section · Sections in this chapter5 / 9 ↓

Sections in this chapter9

Reading tools Definitions & inline links

Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.

Article links

Make inline references clickable

Let's walk through a concrete example of LIME step by step. We'll use a simple three-feature dataset to show exactly how LIME generates explanations.

In[10]:

Code

# Create a simple dataset for demonstration
np.random.seed(42)
n_samples = 200
X = np.random.randn(n_samples, 3)
# Create a non-linear target that depends on features
y = (X[:, 0] > 0.5) & (X[:, 1] < -0.3) & (X[:, 2] > 0.8)
y = y.astype(int)

# Train a complex model
from sklearn.ensemble import GradientBoostingClassifier

complex_model = GradientBoostingClassifier(n_estimators=100, random_state=42)
complex_model.fit(X, y)

# Select an instance to explain
instance_idx = 50
instance_to_explain = X[instance_idx : instance_idx + 1]
true_label = y[instance_idx]
prediction = complex_model.predict_proba(instance_to_explain)[0, 1]

Out[11]:

Console

Instance to explain: [ 0.25049285  0.34644821 -0.68002472]
True label: 0
Complex model prediction: 0.000

We have a specific instance with three feature values. The complex model predicts a probability of belonging to the positive class, and we can compare this with the true label to see if the prediction is reasonable. Now let's implement LIME step by step to understand why the model made this prediction:

In[12]:

Code

# Step 1: Generate perturbations around the instance
n_perturbations = 1000
perturbation_std = 0.5
perturbations = np.random.normal(0, perturbation_std, (n_perturbations, 3))
perturbed_instances = instance_to_explain + perturbations

# Step 2: Get predictions from the complex model
perturbed_predictions = complex_model.predict_proba(perturbed_instances)[:, 1]

# Step 3: Calculate distances and weights
distances = np.linalg.norm(perturbed_instances - instance_to_explain, axis=1)
sigma = 0.5  # Bandwidth parameter
weights = np.exp(-(distances**2) / (2 * sigma**2))

# Step 4: Fit local linear model
from sklearn.linear_model import LinearRegression

local_model = LinearRegression()
local_model.fit(
    perturbed_instances, perturbed_predictions, sample_weight=weights
)

# Step 5: Extract feature importance
feature_importance = local_model.coef_
intercept = local_model.intercept_
local_prediction = local_model.predict(instance_to_explain)[0]

Out[13]:

Console

LIME Results:
Local model coefficients: [ 1.60538105e-37  7.51612588e-37 -5.34329679e-38]
Local model intercept: 0.000
Local model prediction: 0.000

The local linear model gives us coefficients for each feature. These show their relative importance in explaining this prediction. Positive coefficients increase the probability, negative ones decrease it. The intercept is the baseline prediction when all features are zero. Notice how close the local model's prediction is to the complex model's prediction (shown earlier). This closeness tells us the linear approximation captures the model's behavior well in this local region.

Let's verify the quality of our approximation:

In[14]:

Code

import pandas as pd

# Calculate approximation quality
local_predictions = local_model.predict(perturbed_instances)
weighted_mse = np.average(
    (perturbed_predictions - local_predictions) ** 2, weights=weights
)

# Show the most important features
feature_names = ["Feature 1", "Feature 2", "Feature 3"]
importance_df = pd.DataFrame(
    {
        "Feature": feature_names,
        "Importance": np.abs(feature_importance),
        "Direction": [
            "Positive" if coef > 0 else "Negative"
            for coef in feature_importance
        ],
    }
).sort_values("Importance", ascending=False)

Out[15]:

Console

Weighted MSE: 0.0000

Feature Importance Ranking:
     Feature    Importance Direction
1  Feature 2  7.516126e-37  Positive
0  Feature 1  1.605381e-37  Positive
2  Feature 3  5.343297e-38  Negative

The weighted MSE measures how well our linear model approximates the complex model on the perturbed instances. A lower value means a better approximation. Here, the low MSE confirms our linear model captures the complex model's behavior well in this neighborhood.

The feature importance ranking shows which features drive the prediction. Features with larger absolute importance have stronger influence. The direction (positive or negative) tells us whether the feature increases or decreases the predicted probability.