Context Engineering

Context engineering is the practice of deliberately deciding what goes into a language model’s context window on every single inference call. It treats the window as a finite budget to be allocated rather than a bucket to be filled, and it spans everything the model reads at generation time: the system prompt, the tool definitions, retrieved documents, distilled memory, few-shot examples, and the running conversation. Anthropic frames it as the natural successor to prompt engineering: where prompt engineering asks “what words should I write,” context engineering asks “what configuration of information, in what order, at what fidelity, gives the best chance of the outcome I want.” The term moved from niche to mainstream through 2025 and into 2026 as teams discovered that most agent failures are not reasoning failures but context failures: the right token was missing, buried, or crowded out by ten thousand low-signal ones.

Why the Window Is a Budget, Not a Bucket

The intuition that a bigger context window means you can worry less about what you put in it is exactly backwards. Two forces make context a scarce resource even when the nominal window is a million tokens.

The first is context rot: as the token count climbs, recall accuracy on any individual fact in the window degrades. This is not a bug to be patched; it falls out of the architecture. A transformer builds pairwise relationships between every pair of tokens, so an n-token context creates on the order of n squared interactions, and attention gets spread thinner as n grows. Models are also trained on sequences far shorter than their maximum window, so they have fewer specialized parameters devoted to very long-range dependencies. The dedicated context rot entry covers the measurement in detail; the practical consequence is that the hundred-thousandth token makes the first token harder to retrieve.

The second is opportunity cost. Every token you spend on a verbose tool description, a stale file, or a redundant tool result is a token you are not spending on the one code path that actually matters for this step, and it is a token the model still has to attend past. The design goal that falls out of both forces is a single sentence worth memorizing: find the smallest possible set of high-signal tokens that maximizes the likelihood of the desired outcome.

The widget below makes the tradeoff concrete. Adding low-signal tokens to a prompt never helps and usually hurts, because it raises the total token count (pushing you down the context-rot curve) without raising the amount of signal the model can act on.

Try this: hold the high-signal slider still and drag the low-signal slider from 0 upward. The teal dot slides down the curve, recall drops, and the effective signal delivered falls even though you added tokens. That is the whole argument for context engineering in one motion.

The Core Tactics

Context engineering is not one technique but a set of habits applied to each component that lands in the window.

System prompt altitude. The system prompt should sit in a Goldilocks zone: specific enough to steer behavior, general enough to be heuristics rather than a brittle decision tree. A prompt that hardcodes every edge case as an if-then rule breaks the moment an unanticipated case appears and bloats the window in the meantime. Organize it into labelled sections (XML tags or Markdown headers) and aim for the minimal text that fully specifies the expected behavior, which is not the same as the shortest text.

# Scenario: the system prompt for a support triage agent. The left version
# is brittle and long; the right version gives the model room to generalise.

BRITTLE = """
If the ticket mentions 'refund' and plan is 'annual', route to billing-tier-2.
If the ticket mentions 'refund' and plan is 'monthly', route to billing-tier-1.
If the ticket mentions 'bug' and severity is 'high', page on-call.
If the ticket mentions 'bug' and severity is 'low', create backlog item.
...42 more rules...
"""

ALTITUDE = """
<role>You triage inbound support tickets.</role>
<routing>
Route by (a) which team owns the surface area and (b) urgency.
Billing questions with money at stake and an annual plan are higher urgency.
Anything that looks like active data loss or a security issue pages on-call.
When genuinely unsure between two teams, pick the one that can escalate faster.
</routing>
"""

Tool design. Tools should be self-contained, hard to misuse, and clear about when they apply. The common failure is a bloated toolset with overlapping functions, which creates ambiguous decision points the model has to reason through on every turn. A good test: could a new engineer, handed only the tool descriptions, pick the right tool for a task without guessing. Tools should also return token-efficient output; a tool that dumps a 5,000-line file when the agent needed one function has spent the budget badly.

Few-shot examples. Rather than enumerating edge cases in prose, include a handful of diverse, canonical examples that show the shape of good behavior. For a language model an example does the work of a paragraph of description.

Just-in-time retrieval. The older pattern precomputes every possibly-relevant document and stuffs it in the window up front. The emerging pattern keeps lightweight references (file paths, URLs, record IDs) in context and loads the actual content at runtime through tools, the way a person works from a file system rather than memorizing every document. It trades a little latency for a lot of freshness and a much smaller resident context. This is the same instinct behind agentic RAG, where the agent decides what to retrieve as it goes rather than retrieving once at the start.

Long-Horizon Context: Beyond a Single Turn

The tactics above govern one inference call. Agents that run for hundreds of steps need mechanisms to keep context bounded as the transcript grows without number.

# Scenario: a research agent nearing its context limit mid-task. Compaction
# preserves the parts a human would keep in a handoff note and discards the rest.
def compact(transcript: list[dict], recent_keep: int = 8) -> list[dict]:
    head, tail = transcript[:-recent_keep], transcript[-recent_keep:]
    summary = llm.summarize(
        head,
        instructions=(
            "Preserve: decisions made and why, unresolved bugs, current plan, "
            "file paths touched. Discard: full tool outputs, dead ends already "
            "abandoned, anything restated later."
        ),
    )
    return [{"role": "system", "content": f"<summary_of_earlier_work>\n{summary}\n</summary_of_earlier_work>"}] + tail

# Scenario: structured note-taking. The agent keeps a running file so that a
# context reset (or a fresh sub-agent) can pick up exactly where it left off.
def update_notes(notes_path: str, step_result: dict) -> None:
    notes = read(notes_path)
    notes = apply_edits(notes, {
        "## Open questions": step_result.get("new_questions", []),
        "## Decisions": step_result.get("decisions", []),
        "## Next": step_result.get("next_plan", ""),
    })
    write(notes_path, notes)   # lives on disk, not in the window

The subagent orchestration entry covers the coordination pattern in depth, and AI memory covers persistence across whole sessions rather than within one.

Where It Sits Relative to Prompt Engineering and RAG

Context engineering is the superset. Prompt engineering is the part of it that concerns the instruction text. Retrieval-augmented generation is the part that concerns pulling in external documents. Memory systems are the part that concerns carrying state across turns and sessions.

flowchart TD
    A["Inference call: the token budget"] --> B["System prompt<br/>(prompt engineering)"]
    A --> C["Tool definitions"]
    A --> D["Retrieved documents<br/>(RAG)"]
    A --> E["Distilled memory / notes<br/>(memory systems)"]
    A --> F["Few-shot examples"]
    A --> G["Compacted history"]
    B --> H["Context engineering:<br/>allocate the budget across all of these,<br/>every call, for maximum signal per token"]
    C --> H
    D --> H
    E --> H
    F --> H
    G --> H
    H --> I["Model output"]

    classDef default fill:#ffffff,stroke:#4338CA,stroke-width:2px,color:#0F172A,rx:8px,ry:8px;
    classDef data    fill:#EEF0F7,stroke:#0D9488,stroke-width:2px,color:#0F172A,rx:8px,ry:8px;
    classDef process fill:#F7F8FC,stroke:#6366F1,stroke-width:2px,color:#0F172A,rx:8px,ry:8px;
    classDef output  fill:#4338CA,stroke:#4338CA,stroke-width:2px,color:#ffffff,rx:8px,ry:8px;
    class A data;
    class B,C,D,E,F,G,H process;
    class I output;

What’s New (2025 to 2026)

The term itself is the news. Through 2024 the field talked about “prompt engineering” as a catch-all, even when the actual work was retrieval design or memory management. Over 2025, practitioners at Anthropic and elsewhere started using “context engineering” to name the broader discipline, and Anthropic published an engineering guide framing it as the successor practice. By 2026 Gartner had labelled it the breakout AI capability of the year, and a survey literature had formed around it.

Three concrete shifts sit under the rebrand. Just-in-time retrieval displaced front-loading as the default for capable agents, because model tool-use got reliable enough to trust the agent to fetch what it needs. Compaction and external note-taking became standard harness features rather than bespoke hacks; coding agents now ship with them. And sub-agent architectures went mainstream as a context-isolation technique: the point is less about parallelism and more about giving each unit of work a clean, uncluttered window and only paying the coordinator a short summary.

The trajectory also cuts the other way. As models improve, they need less prescriptive context engineering and tolerate more slack in the window. But the finite-resource framing does not go away with scale; it just moves. A model with a ten-million-token window still attends better to a well-curated hundred thousand than to a padded five million.

Practical Checklist

• Write the system prompt at altitude: sections, heuristics, minimal-but-complete. Test it by removing sentences until something breaks.
• Give each agent phase only the tools that phase needs. Audit for overlapping tools.
• Prefer references plus just-in-time reads over front-loaded document dumps.
• Add compaction before you hit the window, not after. Preserve decisions and open bugs; drop raw tool output.
• Give long-running agents an external notes file and make writing to it part of the loop.
• Use sub-agents to keep exploration out of the main window; require them to return short summaries.
• Measure. Build a small eval set and check that a context change actually improves task success, because intuition about what belongs in the window is often wrong.

# Scenario: a coding agent working a long ticket. Every turn you decide
# what goes into the window rather than appending forever. The budget is
# fixed; the job is to spend it on the highest-signal tokens available.
from dataclasses import dataclass

@dataclass
class ContextBudget:
    total: int = 180_000          # model window minus a safety margin
    system: int = 2_000           # stable instructions, cached
    tools: int = 3_000            # only the tools this phase needs
    memory: int = 4_000           # NOTES.md: decisions + open bugs
    retrieved: int = 40_000       # just-in-time file reads for THIS step
    history: int = 60_000         # recent turns, older ones compacted
    # ~71k left as headroom for the model's own reasoning + output

def build_context(budget: ContextBudget, state) -> list[dict]:
    msgs = [{"role": "system", "content": state.system_prompt}]        # altitude: heuristics, not if/else
    msgs += [{"role": "system", "content": render_tools(state.active_tools)}]  # 4 tools, not 40
    msgs += [{"role": "system", "content": state.notes_file}]          # external memory pulled back in
    msgs += compact_history(state.turns, limit=budget.history)         # summarise, keep decisions
    msgs += just_in_time_reads(state.plan_step, limit=budget.retrieved)  # load files now, by path
    assert token_count(msgs) < budget.total - 8_000
    return msgs