A single shared LLM API key used by twelve teams without cost attribution is an accounting bomb. The bill arrives at the end of the month as a single line item, and the conversation that follows: who ran what, which workflow was expensive, and who should be accountable, is painful and inconclusive without data. Enterprise agent platforms need cost attribution built in from the start, not retrofitted after the first surprise invoice. This lesson covers the full stack from per-call metadata tagging through budget enforcement.
Why Cost Visibility Changes Agent Development Behavior
When engineers cannot see the cost of their workflows, they optimize for capability and ignore efficiency. A workflow that issues 15 LLM calls with 4,000-token system prompts per call feels fast and correct in development, where it processes a handful of test documents. In production, processing 10,000 documents per day, the same workflow generates $2,400 in daily LLM spend, a number that surprises everyone when it surfaces on the monthly bill.
Cost visibility changes the feedback loop. When engineers can see a dashboard showing their team’s daily spend against their monthly budget, they make different design decisions: they choose smaller models for classification tasks, cache embeddings rather than recomputing them, and trim system prompts to the minimum effective size.
Tagging Every LLM Call with Metadata
Cost attribution is only as good as the metadata you capture at call time. Retrospective attribution, trying to figure out which team ran which workflow from log timestamps, is unreliable and does not scale. The correct approach is to inject metadata before every API call and persist it alongside the token counts.
import anthropic
from dataclasses import dataclass
from datetime import datetime, UTC
import asyncpg
@dataclass
class CostContext:
team_id: str
workflow_id: str
workflow_type: str
environment: str # "production", "staging", "development"
user_id: str | None = None
# Cost per million tokens (update when pricing changes)
MODEL_COSTS = {
"claude-opus-4-8": {"input": 15.0, "output": 75.0},
"claude-sonnet-5": {"input": 3.0, "output": 15.0},
"claude-haiku-4-5": {"input": 0.8, "output": 4.0},
}
class InstrumentedAnthropicClient:
def __init__(self, db_pool: asyncpg.Pool, cost_ctx: CostContext):
self._client = anthropic.AsyncAnthropic()
self._db = db_pool
self._ctx = cost_ctx
async def create_message(self, model: str, messages: list, **kwargs) -> anthropic.Message:
response = await self._client.messages.create(
model=model, messages=messages, **kwargs
)
# WHY compute cost here rather than in a batch job: immediate persistence
# means budget enforcement can query the cost table in real time without
# waiting for a nightly aggregation to catch overruns.
input_cost = (response.usage.input_tokens / 1_000_000) * MODEL_COSTS[model]["input"]
output_cost = (response.usage.output_tokens / 1_000_000) * MODEL_COSTS[model]["output"]
total_cost = input_cost + output_cost
await self._db.execute("""
INSERT INTO llm_cost_events (
recorded_at, team_id, workflow_id, workflow_type,
environment, user_id, model,
input_tokens, output_tokens, cost_usd
) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10)
""",
datetime.now(UTC),
self._ctx.team_id, self._ctx.workflow_id,
self._ctx.workflow_type, self._ctx.environment,
self._ctx.user_id, model,
response.usage.input_tokens, response.usage.output_tokens,
total_cost
)
return responseThe schema for llm_cost_events should include an index on (team_id, recorded_at) for fast monthly roll-ups and a partial index on environment = 'production' for the dashboards that exclude development noise.
PostgreSQL Cost Aggregation and Budget Enforcement
With events persisted per-call, aggregation becomes straightforward SQL. The budget enforcement query runs before each LLM call and returns the current month’s spend for the requesting team:
async def check_and_enforce_budget(
db: asyncpg.Pool,
team_id: str,
budget_usd: float,
estimated_call_cost: float,
) -> None:
"""Raise BudgetExceededError before an API call if it would breach the monthly cap."""
row = await db.fetchrow("""
SELECT COALESCE(SUM(cost_usd), 0) as month_spend
FROM llm_cost_events
WHERE team_id = $1
AND environment = 'production'
AND date_trunc('month', recorded_at) = date_trunc('month', NOW())
""", team_id)
month_spend = float(row["month_spend"])
soft_warning_threshold = budget_usd * 0.80
hard_cap = budget_usd
if month_spend >= hard_cap:
# WHY raise before the call: the cost is still zero at this point.
# After a 1,200-token response, the overrun is baked in and cannot be undone.
raise BudgetExceededError(
f"Team {team_id} has exhausted monthly budget of ${budget_usd:.2f}. "
f"Current spend: ${month_spend:.2f}. Request manager approval to continue."
)
if month_spend >= soft_warning_threshold:
# Non-blocking warning: emit a metric that triggers a Grafana alert
# without interrupting the workflow. Engineers see the warning in Slack,
# not a production error page.
budget_warning_gauge.labels(team_id=team_id).set(month_spend / budget_usd)Grafana Cost Dashboards
flowchart TD
DB["PostgreSQL\nllm_cost_events"] --> DS["Grafana PostgreSQL\ndata source"]
DS --> MTD["Month-to-date spend\nper team (bar chart)"]
DS --> TRAJ["Spend trajectory\n(actual vs. linear budget)"]
DS --> WF["Cost by workflow type\n(top 10 pie chart)"]
DS --> MDL["Cost by model\n(stacked area chart)"]
MTD --> ALERT["Budget threshold\nannotation lines"]
TRAJ --> ALERT
style DB fill:#EEF0F7,stroke:#6366F1,color:#0F172A
style DS fill:#EEF0F7,stroke:#6366F1,color:#0F172A
style MTD fill:#f0fdf9,stroke:#0D9488,color:#0F172A
style TRAJ fill:#f0fdf9,stroke:#0D9488,color:#0F172A
style WF fill:#f0fdf9,stroke:#0D9488,color:#0F172A
style MDL fill:#f0fdf9,stroke:#0D9488,color:#0F172A
style ALERT fill:#fff7ed,stroke:#f59e0b,color:#0F172AThe spend trajectory panel is the most actionable for budget management. It plots actual cumulative spend alongside a linear projection line (daily budget × days elapsed). When the actual spend line rises faster than the projection, the team is on track to exceed their budget before month end, visible 10 days before the overrun occurs, when there is still time to act.
Showback vs. Chargeback Models
flowchart TD
COSTS["Monthly LLM Spend\nAttributed by Team"] --> SB["Showback Model\nReporting only, no money moves"]
COSTS --> CB["Chargeback Model\nCosts billed to team cost centers"]
SB --> AWARE["Teams see costs,\nbehavior changes gradually"]
CB --> ACCT["Teams directly accountable,\nfaster optimization incentive"]
SB --> RISK["Risk: teams may ignore\nif there is no financial consequence"]
CB --> RISK2["Risk: teams optimize\nfor cost over quality if caps are too tight"]
style COSTS fill:#EEF0F7,stroke:#6366F1,color:#0F172A
style SB fill:#EEF0F7,stroke:#6366F1,color:#0F172A
style CB fill:#EEF0F7,stroke:#6366F1,color:#0F172A
style AWARE fill:#f0fdf9,stroke:#0D9488,color:#0F172A
style ACCT fill:#f0fdf9,stroke:#0D9488,color:#0F172A
style RISK fill:#fff7ed,stroke:#f59e0b,color:#0F172A
style RISK2 fill:#fff7ed,stroke:#f59e0b,color:#0F172AStart with showback: give teams read access to their cost dashboards for one quarter before activating chargeback. This lets them understand their baseline, optimize obvious inefficiencies, and set realistic budget expectations. Teams that discover they are consuming $8,000/month when they expected $2,000 need time to investigate and restructure workflows, not an immediate invoice.
The cost optimization feedback loop that emerges from this visibility is where most enterprise teams find 40-60% savings: identifying that a routing classifier using claude-opus-4-8 for a task that claude-haiku-4-5 handles equally well, discovering a system prompt that grew to 3,000 tokens through accumulated iteration, or noticing that a workflow re-fetches the same reference documents on every call instead of caching them.
The next lesson shifts from cost observability to deployment infrastructure, covering Kubernetes-native patterns for running enterprise agent systems with proper resource limits, health probes, and graceful shutdown handling.