Keep a Team of Agents Running for Months, With Memory a Person Approves

T07-L05 · Agents · Level 5 Operator · 20-minute read, plus 3 hours of doing and 15 minutes a week

A discount nobody offered

A year after it started, the genomics core facility's request desk from T07-L04 has grown. Three agents now run twice a day for about 120 requests a week. One drafts replies to requests. One checks each arriving sample sheet against its request and flags mismatches. One drafts the "your data is ready" message. They share a notes file, so they can remember things like "Group Weber sends RNA in lysis buffer and prefers replies in English."

On a Monday in March, a group leader writes to the facility head: why have three replies in a row offered his group a 10% discount the price list doesn't have? It turns out the intake agent read an email in which a PhD student mentioned "the 10% discount we had at our old facility", and wrote a note: "Group Kaya: 10% discount." Every agent has read that note since, for six weeks. Eleven replies went out with it. The same week, the institute's internal audit asks how AI-drafted replies are produced and checked. And Tobias, who built all of it, starts parental leave in two months.

In a company it's the purchasing team. After a model update, the requisition drafts quietly switched delivery times from working days to calendar days. Nobody noticed for three weeks. Then the auditor for the company's ISO 9001 quality certification asked how the drafts are made.

You are at Level 5 Operator. Many people rely on these agents every day. When they break, work stops or goes wrong for all of them. This book is about running them for months: what you watch, what you do when it breaks, who takes over, and what you can show an auditor.

After this you can

• Decide whether a job needs another agent, and connect agents through files with fixed fields and a script, not through another agent.
• Keep agent memory in a reviewed file that agents may propose to but never write, and switch off memory nobody reviews.
• Log every run and review four numbers each week: volume and failures, quality, cost and drift.
• Handle an incident with a written runbook: stop, find what was affected, correct, tell, and add a test.
• Hand the operation to a deputy who has done the weekly review and one incident drill without you.

Before you start

• Books: T07-L04: the connection, the gate and the connection record this book keeps running. T07-L03 for the agent folders and test sets. T12-L05 for governance records across all your systems.
• Agents in daily use by colleagues, built and connected as in those books, with Claude Code (Anthropic's agent that works on the files in a folder) or OpenCode (a similar agent for other model providers).
• A Git repository for the agent folders, which keeps a history of every change, and a shared spreadsheet for the weekly review.
• A named deputy, agreed with the facility head or your manager, with two hours in their calendar.
• About 3 hours to set things up, then about 15 minutes a week.

Last verified 23 September 2026. Claude Code keeps its own automatic memory on each computer unless you switch it off. Anthropic gives at least 60 days' notice before retiring a model; Claude Sonnet 5 stays available until at least 30 June 2027. Details for experts at the end of this book.

The idea in one page

At Level 4 you connected one agent safely. At Level 5 you keep several running for months, for people who don't know how they work. Five habits make that possible.

Fewer agents than you think. Each agent you add is another test set to maintain, another place for a mistake to hide, and more cost. Add one only when a job has two parts that need different reference files, different permissions or different tests.

Connect agents through files with fixed fields: the intake agent writes a small file with request ID, sample type, amount and mismatches, and nothing else. The coordinator, the part that decides which agent runs next, is a short script, not another agent. A script does the same thing every time and can check each file's fields before passing it on.

Memory is a file a person approves. Agents may propose a note. They never write the notes file itself. Once a week a person accepts or rejects each proposal, and every accepted note carries its source, the date and a review date. Any memory you can't review gets switched off. That includes Claude Code's own automatic memory, which saves notes on the computer where it runs, outside your repository, where nobody sees them.

Watch four numbers every week.

Some things can't wait a week. Set alerts for them: three failed runs in a row, the monthly spend at 80% of its limit, and the mailbox certificate within 30 days of expiry. The coordinator can raise the first two itself, by counting failures and adding up the cost column of the run log, and email the team; the certificate date goes in a shared calendar.

Every change goes through one door. Model, instructions, notes and reference files change only by a commit in the repository, after the check script passes on the full test set, including the planted-instruction tests from T07-L04. Pin the model version: name one fixed model in the settings instead of "the latest", so the model can't change under you. Model retirements come with notice; put the date in the calendar the day it's announced.

Write the runbook before you need it. When something goes wrong, you stop the agents, find every draft affected, correct the cause, tell the people who need to know, and add a test so it can't come back unnoticed. The run log makes the second step possible. Without it you can only guess which drafts were affected.

What stays yours: deciding what the agents may remember, reading the numbers, and making sure someone else can do all of this.

Worked example: the desk through an incident, an audit and a handover

The method comes from T01-L01. Here it's applied to operations, not to a new build. The people, numbers and incidents are examples written for this book.

Lab: the facility's three agents

Structure it. Tobias writes: "The request desk must keep serving about 25 staff and 120 requests a week through my leave and after. Any question from audit, the facility head or a group leader must be answerable from files, not from my memory. A useful result: Anja runs the weekly review and handles an incident without me, and the discount can't happen again unnoticed."

Understand it. He starts with the discount. The run log, one line per agent run, shows every run since the note appeared: 214 runs by the three agents. Because each line records which version of the notes file the run used, he can list exactly which drafts could have seen the note: 38 reply drafts, 11 of them sent. He sends the list to the facility head the same afternoon.

Then he asks how the note got there. When the intake agent was added in the autumn, a colleague gave it permission to edit the notes file, "so it can learn". Nobody reviewed what it learned. It had written 23 notes. Nineteen were fine. Three were outdated. One was the discount.

He also finds a second memory he didn't know about. On the facility server, Claude Code had kept its own automatic memory for each agent folder: 31 short notes in a folder outside the repository. Nobody had read them. Some repeated the reviewed notes; two contradicted them. The agents on Anja's test laptop didn't have them, which explained why her check runs sometimes gave different drafts from the server's.

That changes his problem. Back to Structure it: "The discount wasn't one bad note. Agents could write memory, nobody watched what they wrote, and a second memory existed that nobody knew about. The fix is a rule for memory, a log someone reads every week, and a runbook, before the next incident."

He looks at the agents too. Do they need to be three? The results-notice agent does one thing: it fills a delivery message from the sequencing run sheet. Its drafts are sent unchanged 97% of the time, and the runs never vary. That's a template, not an agent. He replaces it with a fill-in script and one test. Two agents remain.

Choose tools.

• Agents: the two Claude Code folders stay, each with its own test set. The coordinator stays a Python script. He rules out Claude Code's built-in way of letting one agent start others: the order never varies, so a script is easier to test and to log.
• Memory: a file, notes.md, in the repository, which agents may read. Agents may write proposals only to notes-proposed.md. The permission to edit notes.md is removed from every agent's settings, and the check script fails if it ever comes back. Claude Code's automatic memory is switched off in each folder's settings. The alternatives, memory tools such as Letta or mem0, let agents save and recall facts themselves. That's what went wrong here, so he doesn't use them.
• Run log: the coordinator already runs each agent from a script. He adds --output-format json to the claude -p command, which makes Claude Code end each run with a short machine-readable summary that includes the result and an estimated cost. The coordinator writes one line per run to a log file. Langfuse, a tool that records and shows every step of an AI application, is the alternative if the facility ever needs step-by-step detail across many agents. For two agents and 120 requests a week, a log file and a spreadsheet are enough.
• Spend: the server runs on an API key from the Claude Console, Anthropic's account website for paying per use. The key sits in a separate workspace, an area of the account with its own keys, usage and limits, and that workspace has a monthly spend limit set at twice the normal month.

Build it: the memory rule (30 min). The top of notes.md now says who may change it and how:

# Facility notes (agents read this; only staff edit it)
Changes: by commit after the weekly review. Agents propose in notes-proposed.md.
Every note: the fact · source (email date or document) · added by · review by.

- Group Weber sends RNA in lysis buffer; replies in English. · Weber email 2026-02-03 · Anja · review 2026-08
- External customers: prices from prices-external.csv only. · Price list 2026 · Tobias · review 2027-01

He removes the discount note, rechecks the three outdated ones, and adds a line to the reply agent's instructions: "Discounts, dates and prices come only from the reference files, never from notes." He adds a test case in which the notes file contains a discount and the request asks for it. A correct draft quotes the price list.

Then he locks the door in each agent folder's settings file, .claude/settings.json, which Claude Code obeys whatever the instructions say. He adds a deny rule for editing notes.md, and sets automatic memory off with the line "autoMemoryEnabled": false. The check script gets one more test: it reads each settings file and fails if the deny rule is missing or memory is on. On the server he reads the 31 old automatic notes once, then deletes them.

Build it: the run log and the weekly review (60 min). Each log line records the date, agent, request ID, model, the notes and instructions version (the Git commit), cost, number of turns, whether the gate passed, and later, whether the draft was sent unchanged, edited or discarded. It records request IDs, not names. The data protection officer agrees a retention of 12 months, because the log must cover at least one audit cycle.

The weekly review takes 15 minutes on Monday and fills one row of the spreadsheet:

Week 12 shows why the drift column matters. Gate rejections tripled after the new price list. The reviewer opens three rejected drafts: the new list has a column with an ordering email address for each kit, and the agent now copies it into drafts, which the gate's no-address rule catches. One line in the instructions fixes it. The check script passes, and week 13 is back to normal. Without the weekly row, staff would have handled the extra rejections by hand and assumed the agents had got worse.

Proposed notes are reviewed in the same 15 minutes. Most weeks there are two or three; about half get accepted.

Build it: the runbook (30 min). One page, in the repository:

1. Stop. Create a file called STOP in the desk folder; the coordinator checks for it before each run and does nothing while it's there. Tell the team to answer by hand.
2. Find what was affected. Filter the run log by date, agent and notes or instructions version. List every draft, and whether it was sent.
3. Correct. Fix the note, instruction or reference file, by commit, with the check passing.
4. Tell. The facility head always. Requesters who received a wrong reply, with a correction. The data protection officer if personal data went somewhere it shouldn't, the same day. IT if the mailbox connection was involved.
5. Test. Add a case that would have caught it.
6. Record. Date, cause, drafts affected, who was told, the new test. One entry in the incident log.

They run the discount incident through the runbook once more, as it should have gone, with Anja leading. It takes 40 minutes. The eleven groups get a correction the same day.

Build it: updates. The model is pinned with the model line in each folder's settings file, and a second line there makes Claude Code take updates from its stable channel, about a week after release (both lines are in Details for experts). When Anthropic announces a retirement date for the pinned model, the reviewer puts a reminder 30 days before it. A model change is treated like any change: run the full test set on the new model, compare a week of drafts side by side, then switch by commit and watch the drift column for two weeks.

Build it: the audit folder (20 min). The internal audit's question, "how are these replies produced and checked?", gets a one-page index pointing to files that already exist: the connection record and sign-offs from T07-L04, the agent folders and their test sets, the Git history of every change, the run log, the weekly reviews, the notes file with its sources, the incident log, and the data protection officer's retention decision. The auditor asks one follow-up: who checks the checker? The answer is in the weekly review: the spot check of five drafts is done by whoever didn't run the desk that week.

Build it: the handover (2 hours). Anja does the weekly review alone while Tobias watches without speaking. Then he stages an incident on a copy of the desk: a note that tells the agent turnaround is five days. She follows the runbook, stops the copy, finds the four affected drafts from the log, fixes the note and adds a test. She gets stuck once: the runbook says "filter the run log" but not which column holds the notes version. That line goes into the runbook. The facility head signs the operations folder with Anja as operator and a second staff member as her deputy.

What it taught him: the agents needed almost no changes. The operation needed a rule for memory, a log someone reads, and a deputy who had already done it once.

Company: purchasing's agents and the ISO auditor

Structure it. Stefan, the buyer who runs the purchasing agents, writes: "Two agents turn supplier quotes into draft requisitions for eight buyers. The drafts must stay correct across model updates, and the quality management system must show how they're made."

Understand it. He goes back three weeks in the run log to the calendar-days problem. Before the switch, buyers changed the delivery field in 6% of drafts. After a model update in the OpenCode configuration, which someone had set to follow the provider's latest model rather than a fixed version, it rose to 29%. Nobody looked, because the number sat in the log unread. Back to Structure it: "The drafts didn't just get worse. A change arrived without going through the door, and nobody was watching the one number that showed it."

Choose tools. Same as the lab: two OpenCode agents, a coordinator script, a notes file only buyers edit, a run log, a weekly spreadsheet row. The model is pinned to a dated version in each agent file.

Build it. He adds a test case for delivery times in working days, pins the model, and adds a weekly column: share of drafts where a buyer changed the delivery field. He writes the runbook and walks through it with his deputy, Mia. For the auditor, the index points to the same kinds of files as the lab's, filed in the quality management system as a documented procedure with a responsible person, a review date and a change record.

Sign-offs. The works council is informed about the run log, because it records which buyer handled which draft. The head of purchasing confirms in writing that the log is used to check the agents, not to measure buyers. The external data protection officer agrees on 12 months' retention.

In both places the agents stayed small. What kept them trustworthy was a memory a person approved, a log someone read every week, and a second person who could run it all.

What goes wrong

An agent for every idea. Symptom: five agents, five test sets nobody maintains, and one of them is doing a template's job. Fix: each quarter, check whether every agent's runs actually vary. Replace any that don't with a script.

Agents write their own memory. Symptom: a "fact" nobody entered shows up in drafts for weeks. Fix: agents propose, a person accepts in the weekly review, and the check script fails if an agent can edit the notes file.

Memory you didn't know about. Symptom: the server's drafts differ from a colleague's test runs on the same input. Fix: switch off automatic memory in every agent folder's settings, and check the server for leftover memory folders.

The model changes by itself. Symptom: a quality number moves and nobody changed anything. Fix: pin a dated model version, take tool updates from a stable channel, and put retirement dates in the calendar.

A log nobody reads. Symptom: the evidence was in the log for three weeks. Fix: 15 minutes every week, one spreadsheet row, and alerts for what can't wait.

The operator is the documentation. Symptom: a question nobody else can answer while you're away. Fix: runbook, audit index and a deputy who has done the review and a drill without you.

Logs that keep too much. Symptom: a year of request text and names sitting in debug logs. Fix: log IDs, versions, costs and outcomes, not content. Agree a retention period with the data protection officer and delete on schedule.

Do it yourself (3 hours, then 15 minutes a week)

1. Structure it (15 min). Write what must keep working, for whom, through which absence, and which questions must be answerable from files.
2. Understand it (45 min). List every agent, what writes to its memory, and where memory lives, including automatic memory on the server. Check whether each agent's runs vary. Go back through your last incident or complaint and try to list every draft affected. Note what you couldn't find.
3. Choose tools (15 min). Decide the memory rule, what the run log records, where the weekly review lives, the spend limit and the alerts.
4. Build it (90 min). Lock the notes file, switch off automatic memory, add the log line to your coordinator, write the runbook and the audit index, pin the model.
5. Build it: handover (2 hours, with your deputy). They run a weekly review and a staged incident while you watch. Fix the runbook from their questions.
6. Every week (15 min). One row: volume and failures, quality, cost, drift, and the proposed notes.

Exit check

Written finding: an operations folder for agents colleagues depend on, with:

• the run log and weekly reviews for at least four weeks;
• the notes file, with a source, author and review date for every note, and proof that no agent can edit it;
• the runbook, with the record of one drill;
• the audit index, pointing to the connection record, test sets, change history, run log, reviews, incident log and retention decision;
• the handover record: the deputy's name, what they ran, where they got stuck, and what you changed.

It passes when the deputy completes a weekly review and a staged incident without your help, every number in the reviews can be traced to the log, and a question from audit or management is answered from the folder alone.

Rule to remember

Agents may suggest what to remember; a person decides, and someone reads the log every week.

Details for experts

Checked on 23 September 2026. You don't need this to follow the book; it's for setting things up or answering a colleague.

Further reading and tools

Next books

• T07-L04 · Connect an Agent to a Real Mailbox Without Handing It the Keys: the connection and gate this book keeps running.
• T03-L05 · Keep a Production Prompt Healthy for Months, and Hand It Over: growing the test set as incidents add cases.
• T12-L05 · Keep One List of Every AI System Your Team Runs, and Make It Survive You: the same records across every system your team runs.
• T05-L05 · Keep Your Automations Running for Months, and After You Leave: running the scripts and schedules around the agents.
• T13-L05 · Keep Everyone's AI Skills Current, and Show It When Asked: training the colleagues who use the drafts.

Tools

• Claude Code: Taught. The lab's agents, with automatic memory off and a pinned model.
• OpenCode: Taught. The company's agents, with the model pinned in each agent file.
• Git: Taught. The change history for agents, notes and reference files.
• Langfuse: Catalogued. Step-by-step tracing when a log file is no longer enough.
• Letta, mem0: Catalogued. Memory tools where agents save facts themselves; use them only with a person approving what's kept.