A voice pipeline has more moving stages than almost any other production LLM system: ASR streaming partials, an LLM core generating tokens, and a TTS engine synthesizing audio, all within a latency budget measured in hundreds of milliseconds. When something goes wrong, “the call felt slow” is not actionable without the tracing and dashboards this lesson builds.
Tracing a session end to end
Every stage in a turn needs a timestamp on a shared session ID, so a slow or broken turn can be decomposed after the fact instead of reproduced live.
flowchart TB
subgraph SESSION["Session trace (shared session_id)"]
A["ASR partial\nts: caller speech start"] --> B["ASR final\nts: endpoint detected"]
B --> C["LLM first token\nts: dialogue core"]
C --> D["LLM final token\nts: response complete"]
D --> E["TTS first chunk\nts: audio synthesis start"]
E --> F["TTS playback start\nts: caller hears response"]
end
subgraph STORE["Trace store"]
TS[("Time-series DB\nper-stage latency")]
end
subgraph DASH["Control plane"]
DB["Latency / WER / cost\ndashboards"]
ALERT["Stage-level\nalerting"]
end
SESSION --> TS --> DB
TS --> ALERT
style A fill:#EEF0F7,stroke:#6366F1,color:#0F172A
style B fill:#EEF0F7,stroke:#6366F1,color:#0F172A
style C fill:#f0fdf9,stroke:#0D9488,color:#0F172A
style D fill:#f0fdf9,stroke:#0D9488,color:#0F172A
style E fill:#f0fdf9,stroke:#0D9488,color:#0F172A
style F fill:#f0fdf9,stroke:#0D9488,color:#0F172A
style TS fill:#fff7ed,stroke:#f59e0b,color:#0F172A
style DB fill:#fff7ed,stroke:#f59e0b,color:#0F172A
style ALERT fill:#fff7ed,stroke:#f59e0b,color:#0F172A# trace.py - emit one event per stage boundary, same session_id
def emit_trace(session_id: str, stage: str, event: str, ts_ms: int, **meta):
tracer.record({
"session_id": session_id,
"stage": stage, # "asr" | "llm" | "tts"
"event": event, # "partial" | "final" | "first_token" | "first_chunk"
"ts_ms": ts_ms,
**meta,
})
# usage inside the pipeline
emit_trace(sid, "asr", "final", now_ms(), transcript=text)
emit_trace(sid, "llm", "first_token", now_ms(), model="claude-sonnet-5")
emit_trace(sid, "tts", "first_chunk", now_ms(), voice="sonic-2")With this schema, “why was this call slow” becomes a query, not an investigation: subtract adjacent timestamps and the offending stage falls out directly.
What this looks like on a managed platform
If you’re running on a managed platform instead of a self-built pipeline, the same tracing and event data usually arrives as a first-class feature rather than something you instrument yourself. NextNeural, for example, exposes a WebSocket at wss://api.nextneural.ai/ws/live/{call_uuid}?token=nn_k_your_key_here that streams the same shape of stage events this lesson builds by hand:
{ "type": "start", "call_uuid": "cal_c1d2e3f4", "started_at": "2026-05-12T14:32:01Z" }
{ "type": "transcript", "speaker": "agent", "text": "Hi, good morning!", "timestamp": 0.0 }
{ "type": "transcript", "speaker": "customer", "text": "Yes I was looking at that desk.", "timestamp": 6.2 }
{ "type": "end", "outcome": "warm_transfer", "duration_seconds": 113 }And a call.ended webhook delivers the same information as a durable post-call record, with sentiment and a recording URL attached, which is the raw material for the golden eval sets Lesson 19 builds:
{
"event": "call.ended",
"data": {
"call_uuid": "cal_c1d2e3f4",
"outcome": "connected",
"sentiment": "positive",
"duration_seconds": 113,
"transcript": [{ "speaker": "agent", "text": "Hi, good morning!", "timestamp": 0.0 }],
"recording_url": "https://api.nextneural.ai/api/voice/calls/cal_c1d2e3f4/recording"
}
}The tradeoff mirrors Module 1’s build-vs-buy discussion: you get tracing and storage for free, but the stage boundaries are whatever the platform decided to expose, not the ASR/LLM/TTS granularity this lesson’s custom emit_trace schema gives you.
Recording and storing call audio for QA
Call recordings are valuable for QA and eval-set construction, but they’re also the most sensitive artifact the system produces. State a retention policy up front: a bounded window (commonly 30-90 days for QA sampling), automatic PII redaction applied before storage per Lesson 20, and clear caller notice or consent at call start. Store raw audio and transcripts separately with independent access controls, since QA reviewers who need transcripts rarely need raw audio access, and limiting the blast radius of a leaked credential matters here the same way it does for any PII store.
Dashboards for latency, WER, and cost drift
Separate dashboards by stage, not one blended view. A single “voice agent health” panel that combines ASR WER, LLM latency, and TTS cost into one number hides exactly which stage is degrading, the same lesson Module 7’s evaluation lesson applied to conversation quality.
DASHBOARD_PANELS = {
"asr_wer_rolling_7d": "wer",
"llm_ttft_p50_p95_p99": "time_to_first_token_ms",
"tts_ttfa_p50_p95": "time_to_first_audio_ms",
"cost_per_minute_by_stage": "usd_per_min",
"containment_rate_daily": "pct",
}Alerting on stage-level degradation
Two alert types are needed, not one. Absolute thresholds catch sudden breaks (a provider outage spikes p99 latency to 8 seconds, alert immediately). Trend-based alerts catch slow drift (p95 latency creeps up 2% a day for two weeks, crossing no single threshold but adding up to a real regression). Configure both:
def check_alerts(current: dict, baseline: dict, thresholds: dict) -> list[str]:
fired = []
for metric, limit in thresholds.items():
if current[metric] > limit:
fired.append(f"THRESHOLD: {metric} = {current[metric]} (limit {limit})")
drift_pct = (current[metric] - baseline[metric]) / baseline[metric]
if drift_pct > 0.10:
fired.append(f"DRIFT: {metric} up {drift_pct:.0%} vs 7d baseline")
return firedStage-level tracing, retained audio with a real policy, and dual-mode alerting together are what let a team catch degradation on day one instead of day ninety, the same principle Module 8’s cost and scaling lessons build on next.