Capstone: A Production, Self-Improving Support Agent

Every module in this course built one piece of a real Hermes deployment. This capstone puts them back together as one coherent build: a support agent for a small, realistic company, with a phased checklist you can check off, a launch gate that blocks going live until the system earns it, and the stakeholder framing to defend it to someone who has never heard of a tool-call schema.

The Brief

You’re building a support agent for Loomwork, a 40-person SaaS company selling a project-management tool to small agencies. Customers currently reach support through email and a Slack Connect channel; response times lag on evenings and weekends. Loomwork wants an agent that can answer common questions (billing, feature how-tos, account status) around the clock, escalate anything it can’t confidently handle to a human, and get measurably better at its job over the first few months without an engineer manually updating it every week.

Three constraints make this a real build rather than a demo, and each traces back to a specific module.

Customers must never see an internal error message, a half-finished tool call, or a made-up billing figure; a wrong answer here is a support ticket about the support bot (Module 7). The agent needs to be reachable where support conversations already happen, Slack Connect for existing customers, plus a Telegram-based internal escalation channel for the on-call human (Module 6). And the whole point of choosing Hermes over a scripted FAQ bot is that it should get better at handling Loomwork-specific questions over time, without a developer manually adding a new “if this then that” branch every time a new question pattern shows up (Module 4).

Architecture Overview

Every box below is something you configured or built in an earlier lesson. This is assembly, not new engineering.

flowchart TB
    subgraph SURF["Surfaces (Module 6)"]
        SLACK["Slack Connect\n(customers)"]
        TG["Telegram\n(on-call human)"]
    end

    subgraph CORE["Hermes core"]
        MODEL["Model layer (Module 2):\nfast tier for routine Qs,\nfrontier tier + fallback\nchain for hard cases"]
        TOOLS["Curated toolset (Module 3, 7):\nbilling lookup, docs search,\naccount status via MCP"]
        MEM[("Memory + skills (Module 4):\nlearned FAQ patterns,\nescalation history")]
    end

    subgraph OPS["Operations (Module 5, 7, 8)"]
        SANDBOX["Docker terminal backend:\nisolates any tool execution"]
        GATE["Approval gate:\nrefunds, account changes\nroute to human"]
        DOCTOR["hermes doctor +\nsession traces"]
    end

    SLACK --> MODEL
    MODEL --> TOOLS --> MEM
    MEM --> MODEL
    TOOLS -.escalation.-> GATE
    GATE -->|needs human| TG
    MODEL --> SANDBOX
    DOCTOR -.monitors.-> CORE

    style SURF fill:#EEF0F7,stroke:#6366F1,color:#0F172A
    style CORE fill:#f0fdf9,stroke:#0D9488,color:#0F172A
    style OPS fill:#fff7ed,stroke:#f59e0b,color:#0F172A

The one design decision worth calling out: billing lookups and account status run through an MCP server (Module 3) connecting to Loomwork’s actual billing system, read-only, with a tight tools.include filter, rather than as a general-purpose built-in tool. Anything that could change a customer’s account (a refund, a plan change) is deliberately left out of that filter entirely and routes to the Telegram escalation channel instead, per Lesson 20’s approval-gate reasoning.

Build Checklist, Phase by Phase

Phase 3 is the one teams are most tempted to rush under launch pressure, and it’s precisely backwards: an agent that can technically execute a refund because nobody got around to gating it is a launch-blocking risk hiding behind a working demo.

The Launch Gate

Borrow the pattern directly from Module 7, applied across every dimension this capstone touches, not reliability alone:

# capstone_launch_gate.py - run before any go-live decision.
# Every check maps to a phase above; a failure means that
# phase's acceptance criterion has regressed.

def launch_gate(results: dict) -> bool:
    checks = {
        "doctor_clean": results["doctor_exit_code"] == 0,
        "fallback_tested": results["fallback_chain_verified"] is True,
        "toolset_curated": results["disabled_toolsets_covers_destructive_ops"] is True,
        "unapproved_destructive_actions": results["unapproved_destructive_actions"] == 0,
        "escalation_routes_correctly": results["escalation_test_passed"] is True,
        "p95_latency_ok": results["p95_latency_seconds"] <= 6.0,
    }

    failures = [name for name, passed in checks.items() if not passed]

    if failures:
        print("LAUNCH BLOCKED:", ", ".join(failures))
        return False
    print("Launch gate passed.")
    return True

unapproved_destructive_actions is the one line on this list with no acceptable partial credit, exactly like the ACL-violation check in this course’s sibling RAG capstone. A fast, healthy agent that occasionally executes an unapproved refund isn’t a nearly-ready system, it’s an incident that hasn’t happened yet.

Presenting to a Non-Technical Founder

Loomwork’s founder is not going to read this course. Three questions come up in every review like this, and the answers should be ready beforehand.

“What happens if it gets something wrong?” This is where Module 7’s work earns its keep in a conversation, not just in config: the agent can only do the specific things it was explicitly given, looking up an order, answering a documented question. Anything bigger, a refund, an account change, has to ask a human first. A mistake looks like a wrong answer a human can catch and correct, never an action that already happened.

“Why does it get better over time instead of needing constant updates?” This is Module 4’s answer in plain language: the agent remembers patterns in what customers actually ask and builds its own reusable responses to them, the same way a new support hire gets faster after their first few weeks, without an engineer hand-coding a new rule for every question.

“Why not just use a simple chatbot with scripted answers?” Scripted bots handle the questions someone thought to script in advance. This agent handles the ones nobody anticipated, by reasoning over real documentation and account data, while staying inside the same safety boundaries a scripted bot has by construction. You get the coverage of a reasoning agent with the predictability guardrails you’d expect from a simpler system.

What to Build Next

Once Loomwork’s agent is live and the on-call human’s escalation load is trending down week over week, the natural next steps follow the same discipline as the rest of this course: add capability because usage data justifies it, not because it’s available. A second specialized agent for technical/API-integration questions, coordinated via the MCP supervisor pattern from Lesson 22, would make sense once that category of question is common enough to warrant its own accumulated memory. Voice support (Lesson 17) might follow if enough customers ask for it. Neither is worth building on day one, exactly the same cost-benefit discipline you applied to toolset scoping in Phase 2.

The system you’ve now designed across eight modules and one capstone isn’t a finished artifact, it’s a support agent with its own learning loop and a safety boundary built in from the start, which is what lets it get more useful every week instead of drifting into the kind of quiet failure an unguarded agent eventually finds.