Fairness, Explainability & Adverse Action

When decisions are about people, as credit, insurance, hiring, pricing, fraud blocks and benefits all are, getting them right in aggregate isn’t enough. People are entitled to fair treatment and to explanations, and in many jurisdictions the law spells out specific obligations. This lesson covers how to build those into a decision system from the start rather than bolting them on before an audit.

This lesson describes common regulatory themes to inform engineering design. It isn’t legal advice; requirements vary by jurisdiction and product, and change over time. Work with your legal and compliance teams.

Where unfairness enters

Bias in decision systems rarely comes from one place. Each layer of the stack can introduce it:

• Labels: historical outcomes may reflect past discrimination (who was approved, who was policed, who was promoted).
• Selection: training only on people a past policy chose to act on (the selective-labels problem from Lesson 21).
• Features: proxies such as postcode, device type or names can stand in for protected characteristics even when those are excluded.
• Calibration: a model well calibrated overall can be miscalibrated for a smaller group.
• Thresholds and costs: one threshold applied to groups with different score distributions produces different error rates.
• Optimization: an optimizer maximizing margin may concentrate offers, or withhold them, along group lines without anyone intending it.
• Escalation and overrides: human reviewers can introduce their own biases, and who gets escalated can differ by group.

So fairness can’t be certified by inspecting the model alone. It has to be measured on decisions and outcomes.

Measuring fairness at the decision level

Measuring outcomes by group requires knowing group membership. “We don’t collect protected attributes” sounds safe but makes discrimination undetectable. Many organizations keep them (or carefully estimated proxies) in a separate, access-controlled audit store, never as model features, used only for fairness measurement, with an appropriate legal basis.

Common decision-level metrics:

• Action rate parity: the share of each group receiving the action (approval, offer, block).
• Error rate balance: false positive and false negative rates per group (equalized odds).
• Calibration within groups: among people given a score of 0.3, about 30% have the outcome in every group.
• Outcome metrics: for retention offers, the value delivered per customer by group; for pricing, average price paid for comparable products.

import pandas as pd

def fairness_report(df: pd.DataFrame, group: str = "group") -> pd.DataFrame:
    """df: decisions joined to outcomes and (audit-only) group labels.
    Columns: action (0/1), outcome (0/1), score (calibrated prob)."""
    def stats(g):
        pos, neg = g[g.outcome == 1], g[g.outcome == 0]
        return pd.Series({
            "n": len(g),
            "action_rate": g.action.mean(),
            "fpr": neg.action.mean() if len(neg) else float("nan"),
            "fnr": (1 - pos.action).mean() if len(pos) else float("nan"),
            "mean_score": g.score.mean(),
            "outcome_rate": g.outcome.mean(),   # compare to mean_score: calibration
        })
    report = df.groupby(group).apply(stats)
    report["action_rate_ratio"] = report.action_rate / report.action_rate.max()
    return report

Put this on the scorecard (Lesson 19), with confidence intervals for small groups.

The criteria conflict

You can’t satisfy every fairness criterion at once. When base rates differ between groups, a model can’t simultaneously be calibrated within groups and have equal false positive and false negative rates across groups, unless it’s perfect. Different criteria encode different ideas of fairness: equal treatment of equally risky individuals vs equal error burden across groups vs equal access.

Choosing between them is a policy decision, made with legal, compliance and business owners, documented, and revisited. The engineering job is to measure all the relevant metrics, show the trade-offs clearly, and implement the chosen approach.

Mitigation options span the stack: better labels and data collection, removing or transforming proxy features, constraints in the optimization (Lesson 7) such as minimum offer rates per group, and post-processing. Be careful with group-specific thresholds: in some domains and jurisdictions, explicitly using a protected attribute in the decision is itself prohibited. That’s another reason to involve legal early.

Explanations for different audiences

“Explainability” means different things to different people. Design for each:

Reason codes come from two sources: rule hits (Lesson 16), which already have codes, and model drivers. For models, compute per-decision feature attributions (SHAP values are the common choice), map the most negative contributing features to a fixed catalogue of reasons, and translate each to plain language:

REASON_CATALOGUE = {
    "utilization_ratio": ("R10", "Balances are high relative to credit limits"),
    "months_since_delinquency": ("R11", "Recent late payment on an account"),
    "credit_history_months": ("R12", "Length of credit history is limited"),
    "inquiries_6m": ("R13", "Number of recent credit applications"),
}

def top_reasons(shap_row: dict[str, float], k: int = 4) -> list[tuple[str, str]]:
    # Features pushing towards decline (negative contribution to approval score),
    # restricted to features that have approved, customer-facing wording.
    adverse = sorted((v, f) for f, v in shap_row.items()
                     if v < 0 and f in REASON_CATALOGUE)
    return [REASON_CATALOGUE[f] for _, f in adverse[:k]]

Every feature that can drive a decision needs an approved, customer-facing reason. A feature with no honest plain-language explanation is a warning sign, both legally and practically.

Counterfactual explanations tell the person the smallest realistic change that would have changed the outcome: “If your utilization were below 40%, this application would have been approved.” They must use only actionable features (not age, not past events) and respect real constraints. They’re the most useful explanation for the affected person.

Regulatory themes to design for

The details vary, but several themes recur:

• Adverse action notices. In US credit, ECOA and Regulation B require creditors to give specific principal reasons for adverse actions, and regulators have been clear that model complexity is no excuse. Reason codes have to exist by design.
• Rights around solely automated decisions. Under GDPR Article 22, people have rights regarding decisions based solely on automated processing that significantly affect them, including safeguards such as human intervention and contesting the decision. Design human review paths and appeal processes accordingly.
• High-risk AI obligations. The EU AI Act classes some decision uses, such as creditworthiness assessment of individuals and certain life and health insurance pricing, as high-risk, bringing requirements on risk management, data governance, logging, human oversight and documentation. Its obligations phase in over several years, so confirm the current timeline for your use case.
• Model risk management. Banking supervisors expect independent validation, ongoing monitoring and documentation of models used in decisions; in the US, the Federal Reserve and OCC’s SR 11-7 guidance is the long-standing reference.

The engineering implications converge on the same artifacts: decision logs with inputs, versions, reasons and outcomes; versioned models, rules and objectives; documented validation and fairness testing; human review and appeal paths; and monitoring. If you’ve built the system the way this course describes, you have most of them already. The next lesson shows the service that produces them.