Agentic RAG Patterns

Every lesson so far built a progressively better pipeline: one pass, fixed stages executed in a fixed order, and then generation plays whatever hand retrieval happened to deal it. When pass one misses, wrong vocabulary in the query, a genuine multi-hop chain, evidence split across sources that no single search finds together, the pipeline either hallucinates around the gap or refuses outright. 2025 was the year the industry’s center of gravity meaningfully shifted toward the alternative: give the model a search tool directly and let it drive the retrieval process itself, judging the evidence it gets back and searching again until it genuinely has enough. Vendors describe this under various names, agentic RAG, agentic search, deep research, but structurally it’s one consistent idea: retrieval moves inside the model’s own reasoning loop, rather than sitting as a fixed stage before it.

The loop, without framework mystique

Frameworks like LangGraph package this pattern well and handle a lot of useful bookkeeping, but it’s worth seeing first how little is actually mechanically required, before reaching for one of those frameworks. The essential mechanism is nothing more exotic than a tool-calling conversation wrapped in a while loop:

# Agentic RAG in one loop. The "agent" is: a model, one tool,
# stop conditions, and budgets. Everything else is packaging.

SEARCH_TOOL = {
    "name": "search_knowledge_base",
    "description": "Search company knowledge. Returns the top "
        "passages with titles, dates, and IDs. Refine terms and "
        "search again if results miss the target.",
    "input_schema": {
        "type": "object",
        "properties": {
            "query": {"type": "string"},
            "source_filter": {"type": "string", "enum":
                ["any", "confluence", "drive", "jira", "slack"]},
        },
        "required": ["query"],
    },
}

MAX_TURNS, MAX_SECONDS = 5, 30

async def agentic_answer(question: str, user: User) -> Answer:
    messages = [{"role": "user", "content": question}]
    started = time.monotonic()

    for turn in range(MAX_TURNS):
        resp = await client.messages.create(
            model="claude-sonnet-5",
            system=AGENT_SYSTEM,        # grounding + citation rules
            tools=[SEARCH_TOOL],        # from Lesson 16
            messages=messages,
            max_tokens=2000,
        )

        if resp.stop_reason != "tool_use":
            return parse_final(resp)    # model chose to answer

        # Execute the search the model asked for. Note: the SAME
        # two-stage, ACL-filtered retrieval from Module 4. The
        # agent gets no special door around permissions.
        call = next(b for b in resp.content if b.type == "tool_use")
        chunks = await retrieve_and_rerank(
            query=call.input["query"],
            source=call.input.get("source_filter", "any"),
            acl=user.permission_tags,
            final_k=6,
        )

        messages.append({"role": "assistant", "content": resp.content})
        messages.append({"role": "user", "content": [{
            "type": "tool_result", "tool_use_id": call.id,
            "content": format_chunks(chunks),
        }]})

        if time.monotonic() - started > MAX_SECONDS:
            break   # budget cap fired: fall through to best-effort

    # Caps exhausted: answer from what we gathered, honestly.
    return await best_effort_answer(messages)

Read the loop’s real power from the model’s own side of the conversation: it searches “parental leave contractors,” sees results that are actually about full-time employees, and its next search, chosen based on what it just genuinely learned from reading those results, becomes “contractor benefits eligibility policy,” directly informed by the gap it noticed. Compare this against Lesson 13’s HyDE post-mortem: reformulation that’s grounded in actual, real retrieved evidence ages well as a technique; reformulation grounded purely in the model’s own imagination, as HyDE attempted, didn’t age nearly as well, for exactly the reasons discussed there.

The three patterns worth naming

Self-corrective RAG: essentially a pipeline with a safety net bolted on. Retrieve once, exactly as in Module 4; run a cheap grading step that asks “does this evidence actually answer the question?”; on failure, reformulate the query and retry exactly once, then generate regardless of the retry’s outcome. This adds only one extra small-model call, and only on the sad path where the first attempt genuinely failed. If you’re upgrading an existing working pipeline rather than building fresh, this is usually the right starting point: it captures a real, measurable share of the full agentic quality win at almost none of the additional cost or architectural complexity.

Iterative research, the loop shown in full above: built specifically for multi-hop and exploratory questions. “Who manages the team that owns the billing service?” becomes a genuine search-read-search sequence: find the service ownership page first, read out the team name from it, then search the org directory for that specific team’s current manager. This is the graph traversal from Lesson 15, achieved through patient sequential searching rather than through dedicated graph infrastructure, and for occasional multi-hop questions it’s often the more practical choice precisely because it requires no standing index to maintain.

Deep research: the maximal expression of this pattern, popularized widely through 2025: many searches spanning many different sources, several minutes of wall-clock time, producing a structured report with dozens of citations gathered along the way. Structurally it’s the same loop as iterative research, just with substantially bigger budgets and additional explicit planning and synthesis phases layered on top. In enterprise deployments it’s the “compile everything we currently know about vendor X” button, and it needs to be priced and gated accordingly, which is exactly why it must never be the silent default path for an ordinary user question.

flowchart TD
    Q["Query"] --> CLS{"Router\n(same call as rewrite)"}
    CLS -- "simple lookup\n(~80% of traffic)" --> P["Single-pass pipeline\nModules 4-6, ~2s, ~$0.01"]
    CLS -- "multi-hop /\nexploratory" --> L["Agentic loop\n3-5 searches, ~15s, ~$0.05-0.15"]
    CLS -- "explicit research\nrequest" --> DR["Deep research\nminutes, user-visible progress"]
    P --> A["Grounded, cited answer"]
    L --> A
    DR --> R["Structured report"]

    style Q fill:#EEF0F7,stroke:#6366F1,color:#0F172A
    style CLS fill:#fff7ed,stroke:#f59e0b,color:#0F172A
    style P fill:#f0fdf9,stroke:#0D9488,color:#0F172A
    style L fill:#f0fdf9,stroke:#0D9488,color:#0F172A
    style DR fill:#EEF0F7,stroke:#6366F1,color:#0F172A
    style A fill:#f0fdf9,stroke:#0D9488,color:#0F172A
    style R fill:#f0fdf9,stroke:#0D9488,color:#0F172A

Budgets are the deployment decision

Everything that makes the agentic loop genuinely powerful also makes it genuinely expensive: multiplied searches, multiplied tokens consumed across those searches, multiplied wall-clock seconds users have to wait through. The controls that make this pattern actually shippable to real users, rather than a research demo, are boring and non-negotiable: max iterations (3 to 5 is typical for an interactive assistant use case), max tokens across the whole conversation, wall clock limits, and graceful exhaustion behavior, meaning that when a cap fires partway through, the system synthesizes the best answer it can from whatever it gathered so far and says plainly what’s still missing (“I found the service owner but couldn’t confirm the current manager”), rather than a spinner that simply hangs and eventually times out with nothing to show for it. Streaming the agent’s progress to the user (“Searching org directory…”) while this runs does more for perceived quality than an extra loop iteration would, since fifteen seconds with visible activity feels like productive work happening, while fifteen seconds of silence feels like the product is broken.

And it’s worth explicitly noting what the agentic loop did not change, since this is easy to lose sight of amid the added capability: every single search executed inside the loop hits the identical ACL-filtered retrieval pipeline built across Module 4, with no exceptions, and the final answer follows the identical citation contract established in Lesson 16. Agents in this architecture get expanded capability to search iteratively, never expanded privilege to see things a fixed pipeline wouldn’t have shown the same user.

One capability is still genuinely missing from everything built so far in this course: the loop can only ever search what was already indexed by ingestion. The current status of a Jira ticket right now, or a Slack thread posted ten minutes ago, sit outside anything a batch-indexed system can reflect promptly. For those, the agent needs tools that reach directly into live systems, and the 2025 answer to standardizing exactly that kind of tool access is MCP, the Model Context Protocol, which is where the next lesson picks up.