Metadata Filters & Index Routing

Lesson 13 fixed queries that were incomplete. This lesson handles queries that are secretly structured, wearing the costume of natural language while actually encoding a database predicate: “the expense policy changes from last quarter”, “decks Sarah presented this year”, “runbooks in the SRE space”. Every bolded phrase in those examples is a filter condition dressed up as prose. Sending phrases like that straight into the embedding is asking semantic similarity to perform arithmetic it structurally cannot do; extracting them into real filters instead is the difference between “similar-ish documents from whenever, ranked by vague relevance” and “exactly the right documents from exactly the right timeframe, ranked cleanly within that correct set.”

One call, one schema, closed vocabularies

The extraction step runs inside the same fast query-intelligence call established in Lesson 13 for rewriting, since you’re already paying for that round trip and there’s no reason to add a second one just for filters. The non-negotiable design rule governing this extraction, worth treating as a hard constraint rather than a suggestion, is schema-constrained output with closed vocabularies. The model doesn’t get to freely invent filter values from thin air; it picks from a fixed set of values your index actually, verifiably contains:

# Filter extraction with structured output. The enums are
# populated from the index's REAL values at startup, so the
# model physically cannot hallucinate a source that
# does not exist.

from pydantic import BaseModel
from typing import Literal

class QueryFilters(BaseModel):
    # Closed vocabulary: these literals are generated from
    # the actual sources connected in your deployment.
    sources: list[Literal["confluence", "drive", "slack",
                          "jira", "sharepoint"]] = []
    doc_types: list[Literal["policy", "runbook", "contract",
                            "presentation", "ticket"]] = []
    author: str | None = None          # matched against people directory
    updated_after: str | None = None   # ISO date, resolved from
    updated_before: str | None = None  # phrases like "last quarter"
    # Confidence lets the executor decide hard vs soft filtering.
    confidence: Literal["high", "medium", "low"] = "medium"

EXTRACT = f"""Today is {today_iso}. Extract search filters from the
query. Only set a field if the query clearly implies it. Resolve
relative dates against today. If unsure, leave fields empty and
set confidence to low."""

plan = await small_llm.parse(
    system=EXTRACT, user=rewritten_query, schema=QueryFilters,
)

Note the injected Today is {date} line at the very start of the system prompt: relative-time resolution, phrases like “last quarter” or “this year,” is genuinely impossible to get right without it, since the model has no other reliable way to know the actual current date at inference time, and forgetting this single line is by a wide margin the most common bug in real-world implementations of this pattern, exactly as the quiz above walks through.

The extracted filters then ride directly into both retrieval legs from Module 4 as WHERE clauses or engine-specific filter objects, alongside the ACL filter that’s always present on every single query regardless of anything else. Which means your metadata schema is fundamentally a product decision, not merely a technical one: you can only ever filter on what ingestion bothered to capture in the first place. The set that earns its storage cost in almost every deployment worth building: source, doc_type, author, updated_at, space or folder path, and acl_tags. Capture all of them at parse time, as Lesson 4’s ParsedDoc schema already does, because backfilling missing metadata after the fact means re-crawling your entire corpus from scratch, a genuinely painful operation you want to avoid ever needing to run.

Failing soft: filters are preferences, not handcuffs

Extraction will sometimes be wrong, inevitably, and a wrong hard filter produces one of the worst possible user experiences in the entire retrieval stack: zero results returned for a question that was perfectly answerable, with no obvious explanation to the user for why nothing came back. Production systems treat extracted filters as strong preferences with a built-in escape hatch, rather than as absolute constraints:

1. Run the query with all extracted filters applied. Got results? Done, move on.
2. Zero results, and the extraction confidence wasn’t marked high? Retry once, with the most-likely-wrong filters dropped first, dates being the most error-prone category in practice and therefore the first candidate to relax.
3. Still zero results after that retry? At this point it’s honestly a genuine no-evidence case, which the generation stage covered in Module 6 must say out loud to the user rather than silently failing or fabricating an answer.

Log every single step-2 retry event, along with the original query that triggered it. That log stream is a completely free, continuously self-collecting evaluation set for the extractor’s real-world accuracy, and reading through even twenty of these logged retries will teach you more about your actual users’ vocabulary and query patterns than any amount of design-meeting speculation would.

Routing: when one index stops being the answer

Filters select specific rows within a single collection. Sometimes, though, the genuinely correct question isn’t “which rows” but “which collection entirely”:

flowchart TD
    Q["Rewritten query\n+ extracted filters"] --> RT{"Router"}
    RT -- "code-shaped query" --> C1[("Code index\ncode-specialized embeddings,\nfunction-boundary chunks")]
    RT -- "knowledge query" --> C2[("Docs index\ngeneral embeddings,\ncontextual chunks")]
    RT -- "archive explicitly\nrequested" --> C3[("Cold archive\ncheaper storage,\nrelaxed latency")]
    C1 --> M["Merge + rerank\n(RRF handles list fusion)"]
    C2 --> M
    C3 --> M

    style Q fill:#EEF0F7,stroke:#6366F1,color:#0F172A
    style RT fill:#fff7ed,stroke:#f59e0b,color:#0F172A
    style C1 fill:#f0fdf9,stroke:#0D9488,color:#0F172A
    style C2 fill:#f0fdf9,stroke:#0D9488,color:#0F172A
    style C3 fill:#f0fdf9,stroke:#0D9488,color:#0F172A
    style M fill:#f0fdf9,stroke:#0D9488,color:#0F172A

The bar for creating an entirely separate collection should be set deliberately high, because every additional split introduces a new routing decision that can itself be made incorrectly, adding a failure mode that a single well-filtered collection simply doesn’t have. The splits that clear that bar in practice are the ones where the segments are technically genuinely different systems under the hood, not just different topics within the same system: a code index using a code-specialized embedding model paired with function-boundary chunking rather than the prose chunking from Module 3; per-tenant collections deliberately used as a hard security boundary, which Module 7 returns to from the compliance angle; or a multi-year cold archive whose sheer scale would meaningfully drag down the hot index’s query latency if merged in. “Marketing documents versus sales documents” almost never clears this bar on its own; that distinction is simply a filter, handled cleanly within one collection.

When a query does legitimately need to fan out across several genuinely separate collections, you already own the exact tool needed to merge the results back together: reciprocal rank fusion from Lesson 11 fuses ranked lists together regardless of which underlying collection each list originally came from, and the reranker covered in Lesson 12 never cared about collection boundaries in the first place, so it orders the final merged pool exactly as it would order results from a single source.

So far every question type this course has addressed has been answerable purely by finding the right passages. But a question like “who approved the vendors that supply the Berlin office?” isn’t answerable from any single passage at all; it’s a chain that runs across multiple distinct entities and their relationships to each other. That’s the one query class where passage retrieval, however well-tuned, structurally fails, and understanding why that failure happens, along with what to do about it, is exactly why GraphRAG exists as a technique. That’s next.