Permission-Aware Retrieval

Module 1 promised that permissions are precisely why you cannot simply stuff the entire corpus into a prompt and call it done. Here’s the bill for that promise, and it’s a strict one. The requirement is brutal in its simplicity, and worth stating as a single sentence you can hold every design decision in this lesson against: the assistant must never reveal anything the asking user could not have opened themselves, through any legitimate path. Not in a generated answer, not in a citation link, not even in a “here are some related documents” hint offered as a consolation when the main search comes up empty. And this requirement must hold on every single path built across this course so far: the standard pipeline, the agentic loop, the MCP server, and any cache sitting in front of any of them.

The three-layer pattern

Permission-aware retrieval is fundamentally one consistent pattern, applied with equal rigor everywhere it’s needed rather than treated as a one-off feature bolted onto a single endpoint:

1. Capture at ingestion: every chunk carries its source document’s access principals as metadata, set once when the chunk is written.
2. Resolve at query time: the asking user’s identity expands to their current principals, meaning their groups and roles, against a live, authoritative directory, fetched fresh for every single query rather than cached indefinitely.
3. Enforce in the index: the retrieval query itself is filtered so that unauthorized chunks never leave the database at all, never even transiently entering application memory where a bug could expose them.

flowchart LR
    subgraph ING["Ingestion (Lesson 4 captured this)"]
        D["Document + source ACL\n(allowed groups/roles)"] --> CH["Chunks with\nacl_tags metadata"]
    end
    subgraph QRY["Query time"]
        U["User"] --> IDP["Directory / IdP:\nresolve groups fresh"]
        IDP --> F["Filter: acl_tags &&\nuser_principals"]
        F --> IDX[("Filtered hybrid search:\nunauthorized chunks\nnever leave the index")]
    end
    CH --> IDX

    style D fill:#EEF0F7,stroke:#6366F1,color:#0F172A
    style CH fill:#EEF0F7,stroke:#6366F1,color:#0F172A
    style U fill:#EEF0F7,stroke:#6366F1,color:#0F172A
    style IDP fill:#fff7ed,stroke:#f59e0b,color:#0F172A
    style F fill:#fff7ed,stroke:#f59e0b,color:#0F172A
    style IDX fill:#f0fdf9,stroke:#0D9488,color:#0F172A

The load-bearing detail in this whole design is precisely what gets stored on each chunk: stable principals, never expanded users. Store ["grp:finance", "grp:exec"] on the chunk, never the 400 individual usernames currently sitting in those two groups at ingestion time. With that choice made, personnel changes become purely a directory concern, resolved fresh on every single query, and only genuine sharing changes, someone actually granting or revoking a document’s access, ever need to touch the index itself. Get this backwards, baking expanded user lists directly into document tags, and every routine group membership change turns into a mass index update, with access revocation lagging behind on whatever cadence your sync loop happens to run.

In Qdrant, the entire model collapses into one MatchAny filter riding inside the same query as everything else established in Module 4:

from qdrant_client.models import Filter, FieldCondition, MatchAny

# The user's principals come from the IdP at session time:
user_principals = ['user:jkim', 'grp:engineering', 'grp:berlin-office']

results = client.query_points(
    collection_name="chunks",
    query=query_vector,
    # overlap: any shared principal grants read
    query_filter=Filter(
        must=[FieldCondition(key="acl_tags", match=MatchAny(any=user_principals))]
    ),
    limit=40,
)

pgvector expresses the same idea as one array-overlap predicate, WHERE acl_tags && $1::text[], riding inside the same SQL query as everything else, if that’s the path you chose from Lesson 10. Either way, recall Lesson 10’s warning and apply it here directly: benchmark recall with these ACL filters actually turned on, because an ACL that excludes 95 percent of the corpus for a particular user is exactly the highly selective-filter case shown earlier to stress HNSW’s approximate search behavior in ways an unfiltered benchmark would never reveal.

Sync lag is a security property now

Lesson 6 already had you tracking content sync lag as an operational metric. Permissions add a stricter, asymmetric twin worth tracking separately: ACL sync lag, the elapsed time from a permission change happening at the source system to that change being correctly reflected in the index. The two directions of this lag are not symmetric in severity, and treating them as equally urgent is a mistake:

• Grants arriving late are merely an annoyance, a legitimate user briefly unable to see a document they were just given access to, generating a support ticket at worst.
• Revocations arriving late are an active incident, a user who lost access still being able to see content through the assistant that the source system has already locked them out of directly.

So the practical response: subscribe to permission-change events wherever sources offer them, most delta APIs do flag ACL changes distinctly from content changes if you look for that flag, and process revocation events at meaningfully higher priority in the sync queue than everything else, including ordinary content updates. Put ACL lag on the exact same operational dashboard as content lag, but with a noticeably tighter alert threshold given the asymmetric stakes. For the more paranoid tier of deployment, and regulated industries generally belong in that tier, add a verification read at answer time: immediately before final citations render to the user, re-check the top retrieved chunks’ source documents against the live source system’s permission API directly. This costs one batched API call per answer and converts “our sync is usually fast enough” into “we actively checked at the moment of the answer,” which is a considerably stronger sentence to have ready in an audit.

When flat tags run out: ReBAC

Tag overlap models direct sharing relationships well, and it’s genuinely sufficient for a large share of real enterprise permission structures. Real enterprises, though, commonly layer on inheritance and more complex relationships on top: folder trees where sharing cascades down automatically, projects granting access transitively through team membership, delegation chains, and one-off exceptions layered on top of the general rule. Flattening all of that into simple tags at ingestion time is technically possible, computing the effective principal set per document ahead of time, but the flattening logic itself tends to grow into an unauditable shadow reimplementation of whatever permission system the source of truth already maintains, doubling your maintenance burden and creating a second place for permission bugs to hide.

That’s the natural entry point for relationship-based access control engines, systems in the Google Zanzibar lineage such as OpenFGA and SpiceDB. They store the actual relationship graph directly and answer the specific question “can user U read resource R?” by walking that graph at query time, rather than requiring you to pre-flatten every possible relationship into static tags. Two integration shapes work reasonably well for RAG specifically:

• Materialize: the ReBAC engine recomputes effective flat principals per document whenever relationships change, and pushes the results into your chunk tags automatically. Retrieval itself stays a cheap array filter exactly as before; the engine now owns correctness of the underlying expansion logic instead of your ingestion code owning it.
• Check-listwise: retrieval over-fetches a larger candidate set, then a single batched check call against the ReBAC engine prunes that set down to only authorized results before reranking proceeds. Simpler to bolt onto an existing pipeline, but it reintroduces a form of post-filtering with the same fewer-than-k and lag-window caveats discussed earlier, so it belongs behind both a genuinely fast engine and a generous over-fetch margin to stay safe.

Most real deployments live comfortably on materialized tags for a long time; adopt the full ReBAC engine specifically when the organization’s actual sharing semantics already exceed what tags can cleanly express, not preemptively as an architectural flex before that complexity genuinely exists.

One more consumer of this entire design deserves explicit naming, since it’s easy to overlook: caches. A cached answer computed under user A’s specific permission set must never be served back to user B, since their retrieved evidence sets genuinely differ and a shared cache entry would silently leak A’s evidence into B’s answer. Cache keys must therefore always include a hash of the resolved principal set, a detail Lesson 23 returns to directly when caching is covered in full. Permissions now hold firmly on every path built through this course; the next lesson turns to the attacker who writes documents instead of writing queries: prompt injection delivered through your own corpus.