Heidelberg AICurriculum
Track 15 · Advanced
15.4

Runtime context

Compaction, context editing, and a hundred turns that never overflow

7 lessons 2026-08-13 AI-generated

1Overview

Mechanisms that keep a long-running agent inside its context window by summarising, clearing or externalising parts of the conversation as it goes.

Context Engineering is about what you put in. This chapter is about what the system takes out while it is running — automatically, mid-conversation, without ending the session. → Compaction, context editing, and the memory tool: three mechanisms with three different trade-offs, and a clear statement of which are generally available and which are still behind a beta flag. → The measurable outcome is a hundred-turn agent that never overflows and never silently forgets the one thing it needed.

The runtime half of context. Not what you write into the prompt, but what the system removes while the agent is still working — compaction, context editing, and a memory tool the model writes to itself. Three mechanisms, three trade-offs, and a hundred-turn run that survives all of them.

1.2After this chapter you can
→Explain what fills a context window in a long agent run, in order of size
→Use compaction to continue past the window without restarting
→Clear stale tool results and thinking blocks without losing the thread
→Give an agent a memory tool and decide what it is allowed to write
→Measure the token and quality cost of each mechanism
→Tell a generally-available capability from one behind a beta header, and plan for both
1.3When to reach for it

Any agent that runs for dozens of turns or accumulates large tool results — coding agents, research agents, anything with a loop.

1.4Key parts

A compaction trigger and threshold, a policy for which tool results may be cleared, a memory store the model can write to, and a way to see what was dropped.

1.5Free vs paid

These are model-platform features, so the cost is tokens: compaction spends tokens to save tokens and only pays off on long runs.

1.6Watch out

Compaction is lossy by design. What survives is what the summariser thought mattered, and you find out which that was at the worst possible moment.

2Lessons 7

2.1 Identify which parts of an agent run consume the most context

The Info panel displays system prompt and tool schema lines alongside token counts for each block.

Prioritise cuts by ranking context contributors from largest to smallest

  1. Open the transcript of a long run using File Explorer
  2. Search for sections that contain tool output with the Search box
  3. Scroll through the file and highlight blocks labelled as thinking with the Scroll bar
  4. Locate the system prompt and tool schema lines in the Info panel
  • You'll see A token breakdown on a real 60‑turn coding‑agent session where tool results dominate the input tokens
  • Takeaway When an agent stops early, the first suspect is the accumulated tool results rather than the conversation text
  • Check How can you determine which sections of a long‑run transcript are consuming the most tokens before an agent stops early?
  • Cost Free — this is a reading exercise against a transcript you already have.

2.2 Continue past the context window without restarting

The anthropic-beta: compact-2026-01-12 header activates the compaction feature.

Enable compaction so a long run proceeds beyond the token limit instead of erroring or requiring a manual restart

  1. Add the header anthropic-beta: compact-2026-01-12 to every API request
  2. Set the trigger parameter to define when compaction fires, e.g. {"type": "input_tokens", "value": 150000}
  3. Send the request and let the API perform the extra sampling step that creates a compaction block
  4. Read the usage.iterations array in the response and note the separate compaction entry
  5. Sum all entries in usage.iterations to calculate the total tokens billed for the call
  • You'll see The response’s usage.iterations array contains two entries – one compaction entry and one normal message entry – while only a single paragraph of output is shown
  • Takeaway Compaction trades extra tokens for the ability to keep a run alive beyond its original limit
  • Check What header and trigger setting must you add to an API request so that compaction creates a separate usage entry beyond the normal token limit?
  • Cost Real money on every trigger: a full extra model call over (most of) your accumulated context, on top of the turn you actually wanted.

2.3 Clear stale tool results while preserving conversation flow

The context-management-2025-06-27 header enables context editing with strategies like clear_tool_uses_20250919 and clear_thinking_20251015.

Drop old tool payloads and thinking blocks without invoking compaction’s summarisation tax

  1. Enable beta context editing by selecting the context-management-2025-06-27 header
  2. Configure the tool‑result strategy to clear_tool_uses_20250919 and set its parameters such as keep, clear_at_least and exclude_tools
  3. Configure the thinking‑block strategy to clear_thinking_20251015 and adjust its keep setting for the model you are using
  4. Inspect the response’s context_management.applied_edits field to verify how many tool uses and input tokens were cleared
  • You'll see applied_edits reports cleared_tool_uses: 8, cleared_input_tokens: 50000 after a run with many large tool calls
  • Takeaway Context editing acts as a scalpel that removes only the specified tool results and thinking blocks, leaving everything else untouched
  • Check Which beta header and strategy names allow you to clear old tool results and thinking blocks without invoking summarisation during a run?
  • Cost No separate sampling charge — clearing is inline, not a summarization pass. The cost is entirely in what you can no longer see once it’s gone.

2.4 Enable an agent to store facts safely

Adding a tool entry {"type": "memory_20250818", "name": "memory"} to the tools array activates the memory tool.

Persist facts across sessions while restricting what the agent can save

  1. Edit your configuration file and insert {"type": "memory_20250818", "name": "memory"} into the tools array
  2. Implement a server endpoint that processes requests to /memories/* for operations view, create, str_replace, insert, delete, rename
  3. Validate each incoming path so it resolves inside /memories; reject any request that escapes this directory
  4. Add a system‑prompt instruction such as “Only write information relevant to the current project” to define the agent’s writing policy
  5. Restart the application and start a new session to confirm the agent calls view on /memories automatically
  • You'll see A new session starts with Claude issuing a view request on /memories before any other action
  • Takeaway The real value lies in the policy that tells the agent what is worth writing
  • Check What configuration change and system‑prompt instruction let an agent safely store facts across sessions while restricting its write scope?
  • Cost No per-call surcharge for the tool itself; each view/create/str_replace is a normal tool-call round trip, so a chatty memory habit adds turns — and therefore tokens — like any other tool would.

2.5 Compare token usage and recall for each mechanism in a long run

Logging the usage.iterations array distinguishes compaction entries from regular message entries.

Instrument a lengthy run to measure token costs and verify the agent’s memory after compaction or context edits

  1. Select a run that exceeds the compaction (150 K tokens) or context‑editing (100 K tokens) thresholds
  2. Add logging for usage.iterations on every API call and separate sums for entries of type compaction and message
  3. Log context_management.applied_edits and sum cleared_input_tokens across the run to capture saved tokens from context editing
  4. Count memory‑tool calls such as view, create and str_replace to gauge tool round‑trip cost
  5. Later in the same run, query the agent about a detail that was removed by compaction or cleared by a tool result
  • You'll see The agent confidently gives an incorrect answer about information that was cleared three turns earlier
  • Takeaway Token accounting shows runtime cost but not accuracy loss, which you discover by deliberately testing recall
  • Check How do you instrument a long run to compare token usage from compaction, context editing, and memory‑tool calls and then test recall of cleared information?
  • Cost Time, not tokens — this lesson is the one place in the chapter where the expensive resource is your own attention to the transcript.

2.6 Identify stable and beta API mechanisms for release planning

The memory tool (memory_20250818) is listed as generally available in the platform docs.

Create a reference table that shows which Anthropic runtime features are GA and which remain beta

  1. Open platform.claude.com in a browser
  2. Locate the documentation for the memory tool (memory_20250818) and note its status as generally available
  3. Locate the documentation for Compaction (anthropic-beta: compact-2026‑01‑12) and record that it requires a beta header
  4. Locate the documentation for Context editing (anthropic-beta: context-management-2025‑06‑27) and record its beta header requirement
  5. Create a table listing each feature, its version identifier, and whether it is GA or beta
  • You'll see A side‑by‑side table listing the memory tool as GA and compaction plus context editing as beta, each with its required header string
  • Takeaway Distinguish GA from beta because only GA guarantees unchanged behaviour across releases
  • Check Which documentation pages must you consult to classify the memory tool, compaction, and context editing as GA or beta features for a release plan?
  • Cost Free to reason about; the cost of skipping this lesson shows up as a production incident with a confusing root cause months later.

2.7 Choose the right context‑management mechanism for your agent

The keep window and exclude_tools settings control what survives when using context editing, while compaction summarises everything else.

Decide which token‑level strategy to apply and what information must be stored in memory

  1. Restate the warning precisely: compaction summarises, context editing deletes, and only what the summariser or the keep window deems important survives
  2. Select context editing for a tool‑heavy run that is not yet near a hard token ceiling, because it is cheaper and lets you control exemptions via exclude_tools
  3. Choose compaction when the conversation will keep growing regardless of clearing, as it resets the ceiling at the cost of a sampling tax
  4. Use the memory tool for any fact that must survive across restarts or compaction events, writing those facts to memory before they are needed
  5. Combine the approaches: apply compaction to keep active context small and rely on the memory tool for the handful of indispensable facts
  • You'll see You can name, for your own agent, which failure mode belongs to each mechanism and what you have allowed to be lost
  • Takeaway The real decision is what you refuse to lose and write that to memory; everything else may be lossy
  • Check When deciding between compaction, context editing, and the memory tool, what key factor determines which mechanism you should apply to a given agent run?
  • Cost Free — but the decision you make here is the one that determines whether the next compaction event costs you a summarized detail or a fact that was already safe in memory.

3You’ll know it worked 6 checkable outcomes in this chapter

  • ✓Running the graph without overrides uses the defaults and prints them
  • ✓graph.compile succeeds and nodes receive a Runtime object with .context attribute
  • ✓Running the graph logs the expected DB URL and user agent
  • ✓The node prints the overridden value instead of the default
  • ✓graph.invoke without a context argument runs without errors
  • ✓Attempting to use a typed dict without defaults raises an error, while the data class works

4FAQ, Tips & How-to 6

one problem, one solution, one action
How-to Everyone

No default settings in your graph runtime

Create a data class as the runtime context schema so you can set default values for configuration items like LLM provider or DB connection. Data classes allow defaults unlike typed dicts, ensuring required fields are optional unless overridden.

A.I Engineering BootCamp ↗ Lesson → AI-generated
How-to Everyone

Need nodes to read shared data while building a graph

When initializing a LangGraph StateGraph, provide both the state schema and the runtime context schema. This registers the context so nodes can access it via the runtime argument.

A.I Engineering BootCamp ↗ Lesson → AI-generated
How-to Everyone

Need the DB URL and user ID inside a graph node

Annotate the node’s runtime parameter with your context schema type. Inside the node, use runtime.context.field to retrieve values like DB connection or user ID; they are immutable within nodes.

A.I Engineering BootCamp ↗ Lesson → AI-generated
How-to Everyone

Need to change a default value just this time

When invoking the graph, pass a context dict with fields you want to override. Fields with defaults are replaced; required fields must be supplied if not defaulted.

A.I Engineering BootCamp ↗ Lesson → AI-generated
How-to Everyone

Missing defaults make context fields required

If a field in the context schema lacks a default, LangGraph treats it as required and will raise an error unless you supply it at invoke time. Set sensible defaults to avoid mandatory overrides.

A.I Engineering BootCamp ↗ Lesson → AI-generated
Tip Everyone

Switch Between Context Implementations — data class vs typed dict

Typed dicts cannot have defaults and must be accessed with bracket notation; data classes support defaults and dot access. Choose data class for mutable defaults and clearer syntax.

A.I Engineering BootCamp ↗ Lesson → AI-generated

The same set on /recipes, filtered by tool and role.

5See also

Memory is a write problem— durable memory, as opposed to surviving this session —What AI really costs— why a full context window is expensive on every turn —Parallel agent fleets— many contexts instead of one long one —

💬 Discuss this chapter

Ask, share, or report — over on the Heidelberg AI community forum.

Ask a question Stuck on something? The community can help. Show what you built Share it — others can try it and cheer it on. Report a problem or idea Spotted a bug, or have an improvement?