Audit Logging & Compliance

Lessons 19 and 20 built two enforcement mechanisms, RBAC and PII redaction, and both lessons ended by noting that their events should be logged, not just enforced silently. This lesson builds that log properly: what it needs to capture, why rejected and blocked events matter as much as successful ones, and the queries a real compliance review actually runs against it.

Schema: What Belongs in Every Entry

from dataclasses import dataclass, field
from datetime import datetime, timezone

@dataclass
class AuditLogEntry:
    request_id: str
    timestamp: str
    team: str                    # from virtual key metadata, Lesson 19
    key_id: str                  # which specific virtual key, for revocation-scope investigations
    requested_model: str         # the alias the caller asked for
    served_by_model: str | None  # the real deployment that answered, if the request succeeded (Lesson 14)
    outcome: str                 # "success" | "rejected_budget" | "rejected_auth" | "blocked_guardrail"
    detail: str | None = None    # e.g. which PII entity types were found, which budget was exceeded
    cost_usd: float | None = None

def write_audit_entry(entry: AuditLogEntry):
    # Append-only: this store is never updated or deleted in place, only
    # ever appended to, so that altering history requires visibly removing
    # or replacing whole records rather than editing one quietly.
    append_only_store.write(entry)

The outcome field is deliberately broad enough to cover every enforcement point built across this module: rejected_auth for the RBAC scope violations from Lesson 19, rejected_budget for the hard budget caps from Lesson 18, blocked_guardrail for a PII redaction or content-moderation event from Lesson 20, and success for everything that completed normally. Every one of these gets a row.

Why Rejected and Blocked Events Matter Most

A log limited to successful requests would be blind to exactly the events a security investigation cares about most: a key repeatedly attempting to reach a model outside its granted scope, a burst of PII-redaction events from one service (worth investigating even though the redaction worked correctly, because it suggests that service is routinely handling data it maybe shouldn’t be), or a caller repeatedly hitting its budget ceiling in a pattern that looks more like a runaway loop than normal usage. Recording only what succeeded would make the audit log good for cost reporting (Lesson 18) but useless for security review, which is a different purpose the same underlying event stream can and should serve.

The Query a Compliance Review Actually Asks For

-- "Show every request the fraud-detection team made to Anthropic models
-- in the last 90 days, including anything that was rejected or blocked."
SELECT
    request_id,
    timestamp,
    requested_model,
    served_by_model,
    outcome,
    detail,
    cost_usd
FROM audit_log
WHERE team = 'fraud-detection'
  AND requested_model LIKE '%claude%'
  AND timestamp >= now() - interval '90 days'
ORDER BY timestamp DESC;

flowchart TD
    Auth["Lesson 19: RBAC decision\n(success or rejected_auth)"] --> Log["Audit log entry"]
    Budget["Lesson 18: budget check\n(success or rejected_budget)"] --> Log
    Guardrail["Lesson 20: PII/guardrail check\n(success or blocked_guardrail)"] --> Log
    Log --> Store["Append-only store\n(tamper-evident)"]
    Store --> Query["Compliance query:\nby team, model, date range, outcome"]

    style Auth fill:#EEF0F7,stroke:#6366F1,color:#0F172A,rx:8px,ry:8px
    style Budget fill:#EEF0F7,stroke:#6366F1,color:#0F172A,rx:8px,ry:8px
    style Guardrail fill:#EEF0F7,stroke:#6366F1,color:#0F172A,rx:8px,ry:8px
    style Log fill:#4338CA,stroke:#4338CA,color:#ffffff,rx:8px,ry:8px
    style Store fill:#F7F8FC,stroke:#0D9488,color:#0F172A,rx:8px,ry:8px
    style Query fill:#F7F8FC,stroke:#0D9488,color:#0F172A,rx:8px,ry:8px

This query only works because every enforcement point across Module 7 writes to the same schema with consistent team, outcome, and timestamp fields, exactly the discipline of centralizing policy at one chokepoint that has run through this entire course, applied here to the record of what the policy actually did. With RBAC, PII redaction, and audit logging in place, Module 7’s governance layer is complete. Module 8 turns from “was this request authorized and logged” to a different question: “was this request’s answer any good,” instrumenting every model call for tracing, quality monitoring, and controlled experimentation across providers.