Last verified 30 September 2026. All tools here are free. We ran the nightly job with the Gitea runner's local mode on our own server, on one practice task, with our own open model (Ornith 1.5); your results will differ. Details for experts at the end of this book.

The idea in one page

• One rules file for every harness. AGENTS.md in the repository root is read by both DeepSeek Harness and OpenCode. House rules go there, not into each person's settings.
• The test set decides what "done" means. An agent's summary is text it wrote. A job is done when the team's tests pass, including tests the agent didn't write.
• Exit code 0 means the run ended, not that the work is right. Both harnesses stop with an error when the model can't be reached. Neither knows whether the fix is correct. The CI job must run the checks itself.
• Write down every decision a test depends on. If the tests expect one behaviour and the task doesn't say so, a good agent may pick another one.
• Keep three things apart: harness, model, gateway. Each is one setting in the job. Then switching harness is a one-line change, and you can prove the new one is as good by running the same test set.
• An operator's job is the routine. Pinned versions, a monthly rerun of the test set and the safety probe, and a log of what changed.

Set up (steps 1 to 3)

Step 1: Unpack the team kit into a new repository. Unzip the kit, create a private repository on your Git server and push the folder. It contains the labtools practice code, AGENTS.md with three house rules, CHANGELOG.md, the task in .agent/task.md, an acceptance test in ci/test_acceptance.py, the rule check ci/house-rules.sh and the workflow .gitea/workflows/agent.yml. You should see: the repository in your Git server with the workflow listed under Actions (or the equivalent in your CI).

Step 2: Give the job its settings. In the repository's CI settings, add two variables and one secret: GW_URL (your gateway's address ending in /v1), GW_MODEL (the model's name on the gateway) and the secret GW_KEY, a key made only for this job as in T20-L04. You should see: the three names in the settings, the key's value hidden.

Step 3: Check the runner can reach the gateway. Run the workflow once by hand. The first steps install the harnesses at fixed versions and write their settings. You should see: both jobs, one per harness, get past "Point the harness at the team gateway". If a job fails at the agent step within seconds, look for Connection error (address), 401 (key) or 403 (model not allowed for this key).

Worked example: the analysis group's shared agent

Structure it

The group lead's rule: "Anyone may use the agent they like. Every agent's change must follow our house rules and pass our tests before anyone reviews it. Once a night, the agent works through the small-fixes list on its own and leaves a branch." That is three things to build: shared rules, a shared definition of done, and an unattended job.

The group wrote its house rules as three short lines in AGENTS.md, each one something a script can check:

Rules a script can check beat rules like "write clean code": the job can fail on them, and nobody has to argue.

Understand it

Before building the job, the operator checked two things with the T20-L01 bug-fix task and the house rules in place: do both harnesses keep the rules without being told in the prompt, and do they fix the bug?

The prompt didn't mention the rules. Both harnesses loaded AGENTS.md into their very first request to the model (the request grew by about 130 tokens for DeepSeek Harness and 80 for OpenCode, the size of the file plus a heading). In the first run of each, the agent also opened the file with its own tools; in the second, it worked from what the harness had loaded and still quoted the rules while working.

Every run kept every rule. The one failure is the interesting row. The bug report said the plate mean was too low when wells weren't measured. DeepSeek Harness's first run fixed the mean correctly, but kept the unmeasured wells in the plate as empty entries; the acceptance test expected them to be left out. Neither is wrong from the bug report alone. The test encoded a decision the task didn't state, and the agent picked the other option. The second run, same harness, same model, picked the one the test expected.

Two lessons for the operator. First, the same set-up gives different results on different runs, so one run proves little; the test set has to run every time. Second, when an agent "fails" a test, check whether the test asks for something nobody wrote down. The group added one sentence to the task template: "Unmeasured wells are left out of the plate, not stored as empty."

Choose tools

• Shared rules: AGENTS.md in the repository. Personal settings stay personal; anything the group agrees on moves into this file.
• Definition of done: the group's tests plus an acceptance test for each recurring kind of task, and the rule check ci/house-rules.sh, which compares the branch against where the agent started.
• Unattended job: the CI system the group already uses, with the harness installed at a fixed version in each run. The job runs both harnesses side by side for the first month, so the group sees in its own results which one to keep.
• No lock-in: the gateway address, the model name and the key are CI variables; the harness is one line in the job's list. The acceptance tests don't care which harness wrote the code.

Build it

The nightly workflow has five steps: install the harness, write its settings from the variables, run the agent on the task, run the checks, and show what changed, including the new changelog lines and the end of the agent's log. The checks are the heart of it:

If either fails, the job fails, whatever the agent reported. The kit's job stops there and pushes nothing; a last step that pushes a review branch only when every check passed follows the pattern in T08-L05. We didn't test that step here. house-rules.sh fails with a line like RULE 1 broken: no CHANGELOG line, so the person reading the failed job sees which rule and doesn't have to open the agent's log.

The operator ran the workflow twice, both harnesses side by side each time. A whole run took about four minutes.

In the first run, OpenCode did add lines to CHANGELOG.md, but not in the form the rule asks for, and the agent itself reported the work as done. That is the job doing its work: a reviewer would have seen a red job and the rule's number, not a green job with a messy changelog. The first version of the workflow didn't print the changelog lines, so we can't show what OpenCode wrote; the kit's workflow now prints them in its last step, with the end of the agent's log.

Before trusting the job, the operator broke it on purpose three times, once each for the address, the key and the model name:

Both harnesses fail loudly when the model route is broken, which is what a CI job needs. The dangerous case is the other one: the model answers, the harness ends normally, and the work is wrong. Across this track's tests that happened three times with exit code 0: OpenHands failing a feature task's hidden test in our first comparison, OpenCode with a too-small model in T20-L03, and OpenCode's first CI run above. Only the checks caught them.

Understand it, again

After a month of nightly runs, the operator compares the two harnesses on the group's own record: how many nights passed all checks, and what each run cost in tokens on the gateway. The switch is cheap because nothing else depends on the harness: drop the losing one from the job's list and keep the rules, the tests and the gateway as they are.

The same holds for the model. When a better model appears on the gateway, change GW_MODEL, run the test set, and compare. If it doesn't pass the known-answer task from T20-L03, it doesn't go into the nightly job.

Company: one rules file for many teams

The platform team keeps a shared AGENTS.md template with the company's rules (changelog, test placement, no changes to generated files) and each product team adds its own lines below. The platform team owns the job template, pins the harness versions, and publishes the safety probe results from T20-L04 for each version. Product teams choose their harness from the approved list; the platform team checks each new harness version on the shared test set before adding it to that list.

Running it for months

A team set-up decays in four predictable ways. Each needs a small, regular check rather than a big project.

Versions move. Harnesses release often; DeepSeek Harness is still a release candidate and says breaking changes will come. Pin the version in the job, as the kit does. Update on purpose: raise the version on a branch, run the test set and the safety probe, and merge only if both pass. Write the result into the repository's changelog, so the next person knows which version was checked, and when.

The model changes under you. A gateway name like team-coder may point to a new model after the gateway's owner updates it. Ask the gateway's owner to announce changes, and rerun the known-answer task after each one. A sudden change in tokens per night on the gateway can be the first sign.

Rules pile up. Every incident tempts someone to add a line to AGENTS.md. Every line is sent with every request, so a long file costs tokens all night. Keep it short, keep each rule checkable, and move explanations into the README. Once a quarter, delete rules nobody's check enforces.

Tokens and time grow. Watch the gateway's usage per key, and give the job's key a budget, so a runaway night ends in an error instead of a bill. In T20-L02, the two harnesses used very different amounts of tokens for the same results, so the monthly comparison should include tokens, not just passes.

A one-page operator log covers all four: date, harness versions, model, test set result, probe result, tokens per night, and what changed. Keep it in the repository, next to the job.

What goes wrong

• Rules in personal settings. Symptom: agents behave differently depending on who ran them. Fix: move shared rules into AGENTS.md; check them in CI.
• Rules nobody can check. Symptom: "write clean code" in the rules file and arguments in every review. Fix: rewrite each rule as something a script can test, or drop it.
• Trusting exit code 0. Symptom: the nightly job is green and the branch doesn't work. Fix: the job runs the tests, the acceptance tests and the rule check itself.
• Hidden decisions in tests. Symptom: an agent "fails" a test for a choice the task never mentioned. Fix: write the decision into the task template or AGENTS.md.
• One run as proof. Symptom: a harness is chosen or dropped after a single good or bad night. Fix: compare on a month of runs.
• Unpinned harness or model. Symptom: results change and nobody changed anything. Fix: fixed versions in the job; announced model changes on the gateway; rerun the test set after each.
• Harness-specific everything. Symptom: switching harness means rewriting prompts, rules and checks. Fix: rules in AGENTS.md, tasks as plain text files, checks as tests; only one line names the harness.

Do it yourself (2½ hours)

Step 4 · Structure it (20 minutes). Write your group's house rules as three to five lines in AGENTS.md. For each line, write next to it how a script would check it. You should see: no rule without a check.

Step 5 · Understand it (40 minutes). Run the task from the kit twice with each of two harnesses on your own machine, as in T20-L02, with your AGENTS.md in place and without mentioning the rules in the prompt. Record for each run: rules kept, your tests, the acceptance test. You should see: a four-row table like the one above, with your own results.

Step 6 · Choose tools (20 minutes). Adapt ci/house-rules.sh to your rules and decide which tests count as the definition of done. Write the one sentence the task template needs for any decision your tests depend on. You should see: the rule check failing on a branch where you broke a rule by hand, and passing on a clean branch.

Step 7 · Build it (50 minutes). Push the kit with your rules into a repository, set GW_URL, GW_MODEL and GW_KEY, and run the workflow by hand. Then break it on purpose: set a wrong model name and run it again. You should see: a first run with a result per harness, and a second run that fails seconds into the agent step, with a 403 or similar.

Step 8 · Understand it (20 minutes). Start the operator log: harness versions, model, the two runs' results, tokens per run from the gateway, and the date of the next check. You should see: a log a colleague could take over without asking you.

Exit check

Hand in the house rules with their checks, the four-row table from step 5, the link to one passing and one deliberately failed workflow run, and the operator log.

It passes when every rule has a check, the job fails on a broken rule even if the agent reports success, harness, model and gateway are each a single setting, and the operator log names the date of the next test-set run.

Rule to remember

Put the rules where every harness reads them, let tests decide what's done, and keep harness, model and gateway one setting each, so you can switch any of them on evidence.

Details for experts

Checked on 30 September 2026. You don't need this to follow the book; it's for setting things up or answering a colleague.

Further reading and tools

• T20-L04 · Set Up a Harness So a Mistake Stays Small: Container, Rules, Keys and Network
• T20-L02 · Choose a Harness by Testing It on Five of Your Own Tasks
• T08-L05 · Run Automatic Checks, an AI Reviewer and Releases You Can Roll Back
• T07-L05 · Keep a Team of Agents Running for Months, With Memory a Person Approves
• T11-L05 · Keep a Shared Model Server Fast, Affordable and Running
• DeepSeek Harness: Taught
• OpenCode: Taught
• Gitea: Taught
• LiteLLM: Catalogued