▸How-to Scientist +1
Model wanders off topic
The model evaluates evidence against the right question instead of drifting to whichever detail happened to sit near a boundary.
Track 3 · Beginner
3.2
Context Engineering
What the AI sees matters more than how you ask
1Overview
The sequel to prompt engineering: managing what information goes into the model's window and how it's structured. Organised by the Write · Select · Compress · Isolate framework, with anti-pattern → done-right examples that tie into RAG (Dify) and agents (n8n).
This chapter shows how to shape the information presented to an AI so that it stays focused on the right question and avoids drift caused by irrelevant or stale data. By learning techniques such as hypothesis placement, session summarization, targeted retrieval, and strict one‑job prompts, readers will be able to construct lean, high‑signal contexts that keep models accurate across multi‑step workflows.
Context engineering is the sequel to prompt engineering: prompting is how you word the request; context engineering is what information the model sees and how it's organised. Most agent failures come from poor context, not a weak model. The four moves: Write what matters, Select the relevant, Compress the rest, Isolate the concerns.
1.1After this chapter you can
→Decide what to put in the context window — and what to leave out
→Use retrieval (RAG) to pull only the relevant context
→Compress long histories and prune noise to fight context rot
→Isolate concerns across steps and agents for clean context
1.2Why does input matter more than phrasing?
Because the model’s output is driven by the relevance and structure of the data within its context window, so shaping that information determines how well it can answer.
1.3What tools illustrate context engineering in practice?
The chapter shows anti‑pattern vs. correct patterns using Dify for retrieval‑augmented generation and n8n workflows for agent‑based processing, demonstrating real‑world applications.
1.4The moves — weak → strong ladder
- 1Give it the source, not just the question
- 2Instructions at the right altitude
- 3Put key info where the model looks
- 4Retrieve, don't dump (RAG)
- 5Minimal tool set
- 6Just-in-time, not pre-loaded
- 7Compact long history
- 8Strip the noise
- 9Beware context rot
- 10One job per step
- 11Use an external scratchpad
2Techniques
Learn
Write
What you put in the window
Give it the source, not just the question Don't ask the model to recall a specific paper from memory — put the actual text in front of it.
✕Instead of
What does the 2023 Zhang paper say about CRISPR off-target effects?
✓ Try this 💬 AI chat
Paste the paper text (or attach the PDF) and ask: "Using only this paper, summarize what it reports about off-target effects."
Why it works: A model cannot reliably recall one specific paper from training. Give it the real text and it answers from fact, not a fuzzy memory.
Instructions at the right altitude Not a brittle 2-page rulebook, not a vague one-liner — a handful of strong heuristics.
✕Instead of
Be a helpful research assistant. —or— a 2-page list of every edge case and rule.
✓ Try this 💬 AI chat
You are a research assistant for biology students. Be concise, define jargon on first use, prefer primary sources, and say when you are unsure.
Why it works: Over-specified prompts get brittle and hard to maintain; vague ones get generic. A few clear principles steer behavior without micromanaging.
Put key info where the model looks Models attend most to the start and end of the context — not the middle.
✕Instead of
Bury the actual task in the middle of three pages of pasted material.
✓ Try this 💬 AI chat
Instruction at the top, the data clearly tagged in the middle, and restate the ask at the end: "Remember: answer only from the text above."
Why it works: Important instructions buried in a long middle get lost ("lost in the middle"). Anchor them at the edges.
Select
Pull in the right context (RAG)
Retrieve, don't dump (RAG) Pull the few relevant chunks instead of pasting everything.
✕Instead of
Paste all 50 paper abstracts into one prompt and ask your question.
✓ Try this 💬 AI chat
Put the 50 papers in a Dify knowledge base; let it retrieve the 3 most relevant chunks for each question and answer from those.
✕Instead of
Send the whole 80-page protocol manual to answer one question about centrifuge settings.
✓ Try this 💬 AI chat
Index the manual once; retrieve only the centrifuge section at query time.
Why it works: Dumping everything wastes tokens and buries the signal. Retrieval pulls only what is relevant to *this* question.
Minimal tool set Give an agent the few tools it needs, not every tool you have.
✕Instead of
Give your n8n AI agent 20 tools "just in case".
✓ Try this 💬 AI chat
Give it the 3 it actually needs — search papers, fetch a paper, post to chat — each with a clear, distinct name.
Why it works: Too many tools create ambiguous decisions. If a human could not say which tool to use, the agent cannot either.
Just-in-time, not pre-loaded Pass identifiers and let the agent fetch on demand.
✕Instead of
Load the full content of every file into the context up front.
✓ Try this 💬 AI chat
Give the agent file paths / IDs plus a "fetch" tool; it loads a file only when it actually needs it.
Why it works: Pre-loading fills the window with material you may never use. Fetch-on-demand keeps the context lean and relevant.
Compress
More signal, fewer tokens
Compact long history Summarize the conversation instead of re-sending all of it.
✕Instead of
Re-send the entire 60-message chat on every new turn.
✓ Try this 💬 AI chat
Keep a running summary ("decisions made, open questions") and send that plus only the last few messages.
Why it works: Long histories hit context limits and add noise. A running summary preserves what matters in a fraction of the tokens.
Strip the noise Clean the input before the model ever sees it.
✕Instead of
Paste a raw web page — nav bars, ads, cookie banners, HTML — into the prompt.
✓ Try this 💬 AI chat
Extract just the article text first (an n8n node can do this), then send that.
Why it works: Boilerplate dilutes the signal and burns tokens. Pre-cleaning means every token the model reads is relevant.
Beware context rot More context is not always better — recall degrades as the window fills.
✕Instead of
"More context is always better" — stuff the window to the brim with everything that might help.
✓ Try this 💬 AI chat
Keep only high-signal tokens. A focused 4k-token context often beats a sprawling 100k one.
Why it works: As the number of tokens grows, the model's ability to accurately recall any single fact drops. Curate, don't hoard.
Isolate
Separate the concerns
One job per step Split a big task so each step (or n8n node) gets a clean, focused context.
✕Instead of
One mega-prompt that extracts, filters, summarizes, and formats 20 papers at once.
✓ Try this 💬 AI chat
Four n8n nodes — extract → filter → summarize → format — each with a small, focused context.
Why it works: Each step gets a simple context and is independently debuggable — this is context engineering's version of prompt chaining.
Use an external scratchpad Persist state outside the window instead of carrying everything in the chat.
✕Instead of
Try to keep every intermediate result inside the live conversation.
✓ Try this 💬 AI chat
Write intermediate findings to a file / n8n static data / a DB row, and read them back only when needed.
Why it works: External memory keeps long, multi-step tasks coherent without bloating the live context window.
Check yourself
Answer from memory, then scroll up to re-read anything you blank on.
- Why can putting MORE into the context window make an answer worse, not better?
- What problem does retrieval (Select) solve that pasting all your documents into the prompt does not?
- When a chat gets long, why is a running summary better than resending the whole history each turn?
- Why split a big job across separate, focused steps instead of one mega-prompt that does everything at once?
3Lessons 7
3.1 Create a concise session summary for the next model call
A short paragraph that captures the essential facts of the current conversation so the LLM can recall them without excess tokens.
You will be able to produce a one‑paragraph summary that fits within a few hundred tokens and preserves key context for future turns.
- Open a text editor and copy the last five user‑assistant exchanges from your chat log.
- Identify the core facts, decisions, and any hypotheses introduced in those exchanges.
- Write a single paragraph that restates only those core items, omitting filler language.
- Count the words to ensure the paragraph is under 100 words (roughly 150 tokens).
- Save the paragraph as
session_summary.txtfor use in the next prompt.
- You'll see A file containing a brief paragraph that summarizes the conversation without redundant details.
- Takeaway Summarizing history compresses context, preventing older information from being lost when the model’s window fills.
3.2 Summarise a chat turn into a single paragraph
A concise paragraph that captures decisions and evidence from a multi‑turn conversation.
Compress each exchange into a concise paragraph that can be fed back to the model for the next step
- Copy the full transcript of the last user‑assistant turn
- Identify the key decision(s) and any supporting evidence presented
- Write a short paragraph (50‑80 words) that states those decisions and evidence in factual language
- Replace the original turn in your prompt with this paragraph for the next model call
- You'll see The model receives only the summary and produces responses focused on the captured decisions, without drifting into earlier irrelevant details
- Takeaway Summarising conversation history prevents context overflow and keeps the model’s attention on the most relevant information
- Check Which two things must survive into the 50–80 word paragraph that replaces a whole chat turn?
3.3 Build a hypothesis‑anchored prompt for a focused query
A prompt that starts with a clear hypothesis and then asks the model to confirm or refute it using supplied context.
You will craft a prompt that guides the LLM toward a specific answer while keeping the request narrow.
- Read the definition of “hypothesis placement” from the context‑engineering overview (e.g., start with an explicit statement of what you expect).
- Choose a concrete task, such as verifying a refund policy for a product.
- Write a one‑sentence hypothesis like “The company’s return policy allows refunds within 30 days.”
- Follow the hypothesis with a request: “Based on the policy document below, confirm whether this is true and cite the relevant clause.”
- Append the relevant excerpt from the policy document after the prompt.
- You'll see A single prompt that begins with a hypothesis and includes only the necessary policy text for verification.
- Takeaway Anchoring prompts with hypotheses steers the model to evaluate evidence rather than generate unconstrained answers.
3.4 Retrieve only the documents needed for a model query
A filtered set of documents identified by their IDs or paths that are supplied as context to the model.
Fetch and supply just the relevant files so the model answers from that exact information slice
- List all available document IDs or file paths in your repository
- Identify the IDs that contain the required information
- Extract the content of those selected documents only
- Provide the extracted content as the sole context block when calling the model
- You'll see The model returns an answer that cites only the selected document IDs
- Takeaway Limiting context to specific documents narrows the model’s reasoning to relevant data and removes noise
- Check How do you decide which document IDs to extract before you call the model?
3.5 Retrieve only the documents needed for a model query
A targeted retrieval step that selects the minimal set of external files required for answering a specific question.
You will execute a focused search and feed just those results to the LLM, reducing token waste.
- Identify the user’s question (e.g., “What is the current return policy?”).
- From your document store, list all files that might contain policy information.
- Use keyword search (e.g., “return policy”) to filter the list down to the most relevant file(s).
- Open the matched file(s) and extract only the paragraph(s) directly answering the question.
- Save those excerpts to
retrieved_context.txtfor inclusion in the next prompt.
- You'll see A short text file containing just the policy excerpt needed to answer the user’s query.
- Takeaway Targeted retrieval supplies high‑signal data while keeping token usage low, preventing context drift.
3.6 Anchor prompts with a hypothesis
A single‑sentence statement of the central question or hypothesis that frames the model’s evaluation.
Structure prompts so the model evaluates evidence against a specific hypothesis
- Write a concise hypothesis that states the exact question you need answered
- Place this hypothesis as the first line of your prompt
- Add the relevant context after the hypothesis
- End the prompt with the same hypothesis verbatim
- You'll see The response directly addresses the hypothesis and cites only the supplied evidence
- Takeaway Repeating the hypothesis anchors the model’s attention and limits drift
- Check Where does the hypothesis appear in the prompt, and what must stay identical between the two places?
3.7 Compose a strict one‑job prompt using structured output
A prompt that tells the model to perform exactly one task and return results in a predefined JSON schema.
You will produce a prompt that forces the LLM to generate only the required answer in a machine‑readable format.
- Define the single job, e.g., “Extract the refund deadline from the policy text.”
- Create a JSON schema for the output, such as
{ "refund_deadline_days": number }. - Write the prompt: start with system instructions that limit the model to this task only, then provide the retrieved context, and finally request the JSON response.
- Include an explicit instruction: “Do not add any explanation or additional fields.”
- Send the prompt to the LLM and verify that the reply matches the schema.
- You'll see A JSON object containing only the requested field (e.g.,
{ "refund_deadline_days": 30 }). - Takeaway One‑job, structured prompts eliminate ambiguity and make downstream processing reliable.
4You’ll know it worked 20 checkable outcomes in this chapter
- ✓Model lists qualifying abstracts and repeats the hypothesis before each decision
- ✓The dataset is cleaned and rows flagged with reasons, no analysis performed
- ✓The model summarizes only the main content without repeating boilerplate
- ✓The model outputs a 5-tweet thread that only contains content from the provided source
- ✓Screening call outputs a score and one-line reason for each CV without drafting interview questions
- ✓Only DB-RESTORE-02 and NET-FAILOVER-07 sections appear in the window
- ✓Ticket shows only two lines: one for urgency, one for category, with no reply drafted
- ✓The model lists revenue guidance for each quarter and notes any differences from the 10-K figures
20 outcomes in all — one per recipe below.
5FAQ, Tips & How-to 35
one problem, one solution, one action
Research & data tools4
▸How-to Scientist +1
Chat history is overflowing
A tight, factual summary carries forward only the decisions and evidence that matter, cutting context rot that would degrade later answers.
▸How-to Scientist +1
One overloaded prompt trying to do everything
Cleaner outputs at each stage and no confusion from a single overloaded prompt trying to juggle four goals at once.
▸How-to Investor +2
Guidance numbers seem off from the 10‑K
A precise, document-grounded comparison of what management said versus what actually happened, without the model paraphrasing from memory or confusing figures across periods.
Internal tools & ops5
▸How-to Founder +1
When the AI keeps guessing your intent
The model reasons with your principles rather than guessing at intent, and you avoid the "lost in the middle" failure that buries the real rules in verbose instructions.
▸How-to Founder +1
Need answers from specific code files
Faster, more accurate answers because the model works with the specific slice it needs, not a haystack of tangentially-related files.
▸How-to HR / People +1
Want a clean audit of each hiring stage
Each step is auditable and easier to correct; the screening result does not pollute the question-drafting context.
▸How-to Operations +1
Too many runbook steps showing during an incident
The active window stays focused on the current step rather than flooded with procedures that don't apply to this incident.
▸How-to HR / People
Need a concise candidate overview
A candidate brief that references specific achievements and angles from the candidate's real background, making it useful for a hiring-committee prep rather than a boilerplate summary.
Forms, surveys & feedback3
▸How-to Small biz +1
Email chain full of forwarded headers and duplicate text
The model's summary or draft reply is grounded in the real exchange rather than confused by three copies of the same disclaimer block.
▸How-to HR / People +1
Need consistent feedback without a sprawling rubric
Consistent, principle-driven feedback without the model getting lost inside a sprawling rubric where every rule competes for attention.
▸How-to HR / People
Need interview questions that reflect intake notes
Screening questions that match what the client actually said they need, rather than a recycled question bank that could apply to any similar role.
Knowledge & docs2
▸How-to Small biz +1
Need project coherence but don’t want to replay old chat
Long projects stay coherent without burning tokens replaying history; the log is also a useful audit trail.
▸How-to Physician
A focused answer citing the exact guideline clause that applies, with no patient-identifying information in the prompt and no diagnostic or treatment decision made by the model.
Content & marketing2
▸How-to Creator +1
Need a draft that only uses my research notes
Drafts that accurately reflect your actual findings instead of a generic treatment of the topic.
▸How-to Creator +1
Need to draft a post but not publish yet
Each stage does one thing cleanly and there's no risk of an early step accidentally queuing a post before the draft is approved.
Dashboards & analytics3
▸How-to Finance +2
Report begins with filler numbers
Commentary that leads with the right numbers instead of fixating on an early-page figure that happened to be near the top of the context.
▸How-to Finance +2
Can’t recall past budget choices
The model focuses on the new data, not on re-reading old discussions that settled last quarter.
▸How-to Investor +1
Need a source‑cited data‑room summary for an investment memo
A draft memo grounded in the actual data-room documents, with every figure traceable to a named source, rather than a generic template populated with invented figures.
Trackers1
▸How-to Operations +1
When old assumptions creep into a long task
The agent acts on current ground truth and doesn't waste steps (or make mistakes) based on a config value that changed three turns ago.
CRM & sales2
▸How-to Sales +1
Need the exact words from a sales call
Follow-up emails and talk tracks address what the prospect actually said rather than a sanitised version that lost the nuance.
▸How-to Sales +1
Need a quick, focused outreach hook
Outreach drafts that lead with the right hook instead of a generic opener that ignores the recent context.
Customer & client portals2
▸How-to Support +1
Ticket threads full of quoted replies
Accurate routing and faster response drafts because the model reads the live issue, not the full thread history.
▸How-to Support +1
Only urgency and category tags, no reply draft
The draft step never second-guesses the triage decision; the triage step is not polluted by possible-reply options that bias its classification.
▸How-to Everyone
Web pages full of ads and navigation bars
Clean the input before the model ever sees it. Boilerplate dilutes the signal and burns tokens. Pre-cleaning means every token the model reads is relevant.
▸How-to Everyone
Too much context in prompt
More context is not always better — recall degrades as the window fills. As the number of tokens grows, the model's ability to accurately recall any single fact drops. Curate, don't hoard.
▸How-to Everyone
Chat history gets too long
Summarize the conversation instead of re-sending all of it. Long histories hit context limits and add noise. A running summary preserves what matters in a fraction of the tokens.
▸How-to Everyone
Keep the conversation tidy
Persist state outside the window instead of carrying everything in the chat. External memory keeps long, multi-step tasks coherent without bloating the live context window.
▸How-to Everyone
One huge prompt trying to handle many papers
Split a big task so each step (or n8n node) gets a clean, focused context. Each step gets a simple context and is independently debuggable — this is context engineering's version of prompt chaining.
▸How-to Everyone
Loading every file wastes space
Pass identifiers and let the agent fetch on demand. Pre-loading fills the window with material you may never use. Fetch-on-demand keeps the context lean and relevant.
▸How-to Everyone
Too many paper abstracts in a prompt
Pull the few relevant chunks instead of pasting everything. Dumping everything wastes tokens and buries the signal. Retrieval pulls only what is relevant to *this* question.
▸How-to Everyone
Agent swamped by tool choices
Give an agent the few tools it needs, not every tool you have. Too many tools create ambiguous decisions. If a human could not say which tool to use, the agent cannot either.
▸How-to Everyone
When my prompt is too detailed or too vague
Not a brittle 2-page rulebook, not a vague one-liner — a handful of strong heuristics. Over-specified prompts get brittle and hard to maintain; vague ones get generic. A few clear principles steer behavior without micromanaging.
▸How-to Everyone
Key instructions get lost in a long prompt
Models attend most to the start and end of the context — not the middle. Important instructions buried in a long middle get lost ("lost in the middle"). Anchor them at the edges.
▸How-to Everyone
Model can’t recall a specific paper
Don't ask the model to recall a specific paper from memory — put the actual text in front of it. A model cannot reliably recall one specific paper from training. Give it the real text and it answers from fact, not a fuzzy memory.
The same set on /recipes, filtered by tool and role.
6Videos 2
Stop Vibe Coding: Context Engineering & RAG for AI Agents (ft. Cole Medin)
Cole Medin on why context — not a bigger model — is what makes agents work, with RAG shown in practice.
Stop Vibe Coding: The 10x Context Framework That Actually Works (Featured by Cole Medin)
A concrete framework for feeding an AI coding assistant the right context up front.
7FAQ 7
What is context engineering, in one sentence?
It is managing what information goes into the model's context window — what to include, what to retrieve, what to compress, and what to keep separate — so the model has exactly what it needs and little else.
Why does a strong model still give bad answers?
Usually because of poor context, not a weak model: the key fact was missing, buried, or drowned in irrelevant text. Most agent failures trace back to what the model could see that turn, not its raw ability.
What is RAG and how does it fit in?
Retrieval-augmented generation: instead of hoping the model remembers, you fetch the relevant documents and put them in the context, so it answers from the actual text. It is the "Select" move of context engineering, and it is what tools like Dify do under the hood.
My chat gets worse the longer it runs — why?
That is context rot. As the window fills with old turns and side-tracks, the signal-to-noise ratio drops and the model loses the thread. Summarise what matters, drop what does not, or start a fresh session with just the essentials.
Does a bigger context window solve everything?
No. More room helps, but quality still degrades as you fill it, and models reliably miss facts buried in the middle ("lost in the middle"). A big window is not a reason to dump everything in — selection and compression still pay off.
Where should I put the most important information?
Near the start and the end of the context. Models attend least to the middle, so a critical instruction or fact stranded in the middle of a long prompt is the most likely thing to be ignored.
How does this connect to agents and automation tools?
Agents are context engineering in motion: each step needs a clean, focused context, and good agent design isolates concerns so one step's clutter does not pollute the next. Retrieval nodes in n8n or Dify are the "Select" and "Compress" moves applied in a pipeline.
8Glossary 9 terms
Show the 9 terms
The four moves
Write- Put what matters into the context window — instructions, the source text, the task.
Select- Pull in only the relevant context (retrieval / RAG) instead of dumping everything.
Compress- More signal, fewer tokens — summarise histories, prune noise to fight context rot.
Isolate- Separate concerns across steps and agents so each gets a clean, focused context.
Concepts
Context window- Everything the model can see in one turn. It is finite — tokens cost money and quality degrades as it fills.
RAG- Retrieval-augmented generation — fetch the relevant documents and put them in the context so the model answers from fact, not memory.
Context rot- The drop in answer quality as the window fills with stale or irrelevant text — the model loses the thread.
Lost in the middle- Models attend most to the start and end of the context and can miss key facts buried in the middle.
Token- The unit of text a model reads and bills by — roughly ¾ of a word.
9See also
💬 Discuss this chapter
Ask, share, or report — over on the Heidelberg AI community forum.