SHAP Explainability in Transaction Monitoring: Making Your ML Model Audit-Ready

When a fraud model blocks a transaction worth ₹80,000, a customer calls the bank. The analyst needs to explain — clearly, in plain language — why the model flagged it. "The model scored it 0.87" is not an explanation.

This is the explainability problem. It is not merely a UX concern. RBI's model governance guidelines explicitly require that ML-based fraud decisions be explainable to auditors and, on request, to customers. SHAP (SHapley Additive exPlanations) is the most rigorous solution — but applying it correctly to a production scikit-learn pipeline has non-obvious steps.

This article walks through how vcurd implements SHAP-based explanations on top of the XGBoost + IsolationForest pipeline, what the output looks like, and how to make it audit-ready.

What SHAP Actually Measures

SHAP values answer one question: for this specific prediction, how much did each feature contribute to the final score, relative to the model's average prediction?

A feature with a SHAP value of +0.3 pushed the fraud score up by 0.3 from baseline. A feature with −0.15 pushed it down. The sum of all SHAP values plus the baseline equals the model's raw output for that prediction.

This is fundamentally different from feature importance — which tells you what the model uses globally — and different from rule-based explanations — which are deterministic but don't account for feature interactions. SHAP is local (per-prediction) and model-agnostic.

Integrating SHAP with a scikit-learn Pipeline

The challenge in production fraud ML is that models are usually wrapped in scikit-learn Pipelines with preprocessing transformers. SHAP's TreeExplainer works on the raw XGBoost model, not the Pipeline — so you need to extract transformed features, then compute SHAP values on the XGBoost step alone.

import shap
import numpy as np

# pipeline = FeatureEngineeringTransformer → IsolationForestTransformer → XGBoost
# Separate transform from predict steps

def explain_transaction(pipeline, transaction_df):
    # Run all steps except the final classifier
    preprocessing_steps = pipeline[:-1]
    X_transformed = preprocessing_steps.transform(transaction_df)

    xgb_model = pipeline[-1]  # Final XGBoost step
    explainer = shap.TreeExplainer(xgb_model)
    shap_values = explainer.shap_values(X_transformed)

    feature_names = preprocessing_steps.get_feature_names_out()

    return {
        "baseline": float(explainer.expected_value),
        "prediction": float(xgb_model.predict_proba(X_transformed)[0][1]),
        "shap_values": dict(zip(feature_names, shap_values[0].tolist())),
    }

The key insight: extract the transformation pipeline up to (but not including) the XGBoost estimator, transform the input, then pass the already-transformed data to TreeExplainer. This avoids the mismatch between Pipeline input and what XGBoost actually sees.

Which Features Matter Most in UPI Fraud?

Running SHAP across vcurd's training dataset reveals the feature importance hierarchy for UPI fraud detection:

velocity_1h — transactions from this account in the past hour — is consistently the strongest predictor. A legitimate account rarely sends more than 3 UPI transactions per hour; fraudsters use compromised accounts to drain funds in bursts of 8–20 transactions before the account is locked.

Turning SHAP Values into Human-Readable Reasons

Raw SHAP values are not suitable for customer-facing explanations. The LLM layer in vcurd takes the SHAP output as structured input and generates a natural-language reason:

def build_shap_prompt(shap_output, transaction):
    top_features = sorted(
        shap_output["shap_values"].items(),
        key=lambda x: abs(x[1]),
        reverse=True
    )[:3]

    prompt = f"""
Transaction flagged. Fraud probability: {shap_output['prediction']:.2f}
Amount: ₹{transaction['amount']:,.0f}
Top fraud signals:
{chr(10).join(f'- {k}: SHAP={v:+.3f}' for k, v in top_features)}

Write 2-3 sentences explaining why this transaction was flagged,
in plain language suitable for a bank analyst. Be specific about
which signals raised concern. Do not use jargon.
"""
    return prompt

Example output for a transaction with high velocity + new beneficiary + ₹45,000 amount:

Making It Audit-Ready: What to Log

For RBI model governance compliance, every ML-scored transaction should persist the following to your audit log:

1. Raw model output: The probability score (e.g., 0.87), not just the final decision.
2. Top 5 SHAP values: Feature name + value, sorted by absolute magnitude.
3. Model version: The exact model artifact hash or version ID that produced this score.
4. Feature values at scoring time: Snapshot of the engineered features (velocity, anomaly score, etc.) as they were when the model ran.
5. Human-readable explanation: LLM-generated text, logged alongside the structured SHAP output.

In vcurd, this is all written to the rule_logs table alongside the rule engine output, keyed to the transaction ID. The export endpoint at GET /analytics/audit-export packages this into a format that maps to RBI's model audit template.

Limitations and When SHAP Is Not Enough

SHAP explains what the model saw — it does not verify causation. A transaction can be flagged for high velocity because the account holder was genuinely busy (legitimate). SHAP tells you the model's reasoning; human review determines the correct outcome.

For this reason, vcurd's explainability layer is always paired with the rule engine — deterministic rules provide hard audit trails that SHAP values alone cannot. The combination gives you both the interpretable model output (for auditors) and the deterministic rule log (for legal teams).