Choose a Harness by Testing It on Five of Your Own Tasks

T20-L02 · AI harnesses · Level 2 Power user · 15-minute read, plus 2 hours of doing

Two tools, one recommendation

Your group asks you which coding assistant to standardise on. Half the lab uses OpenCode; a postdoc swears by DeepSeek Harness since it came out. In a company it's the analytics team: two harnesses in use, one budget and a manager who wants one name by Friday.

You could read reviews, but they test someone else's tasks on someone else's model. In T20-L01 · Same Model, Different Harness: See What the Program Around the AI Changes you saw that the same model behaves differently inside different harnesses, and that one easy task tells you little. In this book you set up a fair test on your own computer: two harnesses, one free model, five tasks from your own work, each with an answer only you know. At the end you have a table and a five-line recommendation your group can check. You are at Level 2 Power user: your team will follow your pick, so the test has to hold up.

After this you can

• Run DeepSeek Harness next to OpenCode Desktop, both on the same free model.
• Turn five real tasks into test cases, each with a known answer kept away from the agent.
• Run every task in both harnesses on separate copies, and score each run the same way.
• Read a result table and write a recommendation that names its limits.

Before you start

• Books: T20-L01 · Same Model, Different Harness: See What the Program Around the AI Changes and T08-L02 · Get a Script Your Colleague Can Rerun, Without Being a Developer, where you checked a script against an answer worked out by hand.
• OpenCode Desktop from T08-L01 · Start With OpenCode: Set It Up, Change One Line, and Undo It.
• A computer you may install software on, and an email address for an OpenCode Zen account.
• Five small tasks from your own work that a coding assistant could do in one go, with files you may share with a cloud model. The practice folder from T20-L01 works for the first run.
• About 2 hours 15 minutes: 15 to read, 30 to set up, 90 for the runs.

Last verified 30 September 2026. DeepSeek Harness and OpenCode are free. OpenCode Zen gives both the same free model, Space Bunny, which keeps none of your prompts; it's free for a limited time. Zen asks for billing details when you create a key. DeepSeek Harness is a developer preview. Details for experts at the end of this book.

The idea in one page

• Keep everything fixed except the harness. Same model, same task text, same starting folder. If anything else changes, you are testing that instead.
• Your tasks, not a benchmark. A harness that wins on public tests can still stumble on your file formats and habits.
• Every task needs a known answer. Before the run, write down what a correct result looks like: a number the script must print, a file that must appear, a line that must change. Keep it outside the project folder.
• One copy per run. Each harness gets its own fresh copy of the folder. Two agents in one folder overwrite each other's work, and one can read the other's.
• Score the same way every time: passed the known answer, minutes, and three habits: did it show the fault first, did it check its own work, and did its final message match what it did.
• Five tasks, then decide. One task is luck. Five from your own work show a pattern you can defend.

Set up (steps 1 to 4, once)

Step 1: Install Node.js. Go to nodejs.org, download the version marked LTS and install it with the default choices. Node.js is the program DeepSeek Harness runs on. You should see: in a new terminal (PowerShell on Windows, Terminal on a Mac), node --version prints v24 or higher.

Step 2: Get an OpenCode Zen key. Go to opencode.ai/zen, sign in, add your billing details as the page asks, and copy your API key into a password manager. The free models cost nothing; the key lets a harness other than OpenCode use them. You should see: your key, and Space Bunny Free listed as free in Zen's model list.

Step 3: Start DeepSeek Harness in a practice folder. Make a folder harness-test in Documents with two subfolders, dsh and opencode. Unzip the T20-L01 practice folder into each. In the terminal, go into harness-test/dsh/labtools, switch off DeepSeek Harness's feedback upload, and start it with a pinned version:

$env:DSH_TELEMETRY_MODE="DISABLED"     # Windows PowerShell
export DSH_TELEMETRY_MODE=DISABLED     # Mac
npx @deepseek-ai/dsh@0.2.0-rc.2 web

The first line depends on your system; the last line downloads and starts DeepSeek Harness for the folder you're in. You should see: your browser opening DeepSeek Harness at 127.0.0.1:3080.

Step 4: Give DeepSeek Harness the same model as OpenCode. Open Settings → Models, choose Add model provider, then switch the card to Custom model API. Fill in Provider ID zen, base URL https://opencode.ai/zen/v1, API protocol OpenAI Chat Completions, your Zen key, and the model space-bunny-free. Save, then pick space-bunny-free in the model picker. You should see: space-bunny-free as the selected model in DeepSeek Harness. In OpenCode Desktop, check that the model is Space Bunny too.

Worked example: five plate-reader tasks, two harnesses

Structure it

The group's real question: which harness should we use for small script changes? Five tasks from the lab's own plate-reader tool, from easy to messy:

1. Bug fix: wells A10 to A12 land in the wrong column, and unmeasured wells pull the mean down.
2. New option: --min-od, with a README line.
3. Grid files: accept plates saved as an 8-by-12 grid as well as a list.
4. Blank correction: --blank A1,A2 subtracts the blank wells' mean.
5. New reader: files with info lines on top, semicolons and decimal commas.

Each task got a known answer, kept in a separate folder: small test files and the exact number the tool must print for each.

Understand it

Before the runs, the known answers were checked twice: the untouched tool fails them, and a correct fix written by hand passes them. An answer the original tool already passes can't tell you anything, and an answer that even a correct fix fails is a broken test.

Choose tools

DeepSeek Harness and OpenCode, same model, same task text, same starting folder, one fresh copy per run. The measures: known answer passed, minutes, and the three habits.

Build it

Each harness ran each task once, on 30 September 2026, with the same open model on our own hardware instead of Space Bunny, so the runs could go unattended. Then the known answers were checked.

Understand it: read the table

Both passed every known answer, so passes don't decide here. The other columns do. OpenCode used about half the tokens and two thirds of the time, because it asked the model less often: 56 requests against 95. DeepSeek Harness was the more careful one. In four of the five tasks it ran the existing tests before changing anything (OpenCode did so once), in the bug fix it proved its new tests catch the fault, and on the grid task it tried a layout nobody had asked about. It also spent four steps on that task looking for a test program that was already installed.

The recommendation the group got, in five lines:

1. Use OpenCode for small script changes with our model.
2. Deciding score: equal passes (5 of 5 each), about half the tokens, two thirds of the time.
3. Habit that mattered: DeepSeek Harness checks more. For bug reports, ask OpenCode to show the fault before fixing it.
4. Not covered: changes across many files, other models, and more than one run per task. A published test on DeepSeek's own model found the token gap the other way round.
5. Test again when we change model, or when either harness has a major release.

Company: the returns report

The analytics team does the same with its own five tasks: a date-format bug, a new filter, a second export layout, a currency conversion and a file from a new shop system. Known answers are totals the team already reconciled by hand last quarter. Two copies of the project per task, one per harness, and the same table.

What goes wrong

• Two harnesses in one folder. Symptom: odd leftovers, or one run "knows" things it shouldn't. Fix: one fresh copy per harness per task; delete it after scoring.
• Different models by accident. Symptom: one harness is far better at everything. Fix: check the model name before every run. DeepSeek Harness starts on DeepSeek's own model until you pick another.
• The answer is in the folder. Symptom: a perfect run whose steps include searching for files you never mentioned. Fix: keep the known answers in a different folder, as in T20-L01 · Same Model, Different Harness: See What the Program Around the AI Changes.
• Only easy tasks. Symptom: both pass everything and you still don't know which to pick. Fix: include at least one task an assistant has got wrong for you before.
• The free model changes under you. Symptom: runs on Monday and Thursday differ for no clear reason. Fix: run both harnesses on a task back to back. If Space Bunny goes away, pick another free model that keeps no data and rerun both.
• An update mid-test. Symptom: a harness behaves differently halfway through. Fix: pin the version, as in step 3, and write it in your table.

Do it yourself (90 minutes)

Step 5 · Structure it (20 minutes). Write your five tasks as short requests, each ending with "Add tests. Run the test suite before you finish." For each, write the known answer in a file answers.txt in harness-test, outside both copies. You should see: five requests and five answers, each answer a number, a file or a line you can check without the agent.

Step 6 · Understand it (10 minutes). For each task, check the answer against the untouched files: it must fail now. If it already passes, sharpen the task or the answer. You should see: five answers that the untouched folder fails.

Step 7 · Build it (50 minutes). For each task, make two fresh copies of the task folder, one in dsh and one in opencode. Stop DeepSeek Harness with Ctrl+C in its terminal, go into the new dsh copy and start it again as in step 3; add the other copy to OpenCode Desktop as a project. Paste the same request into both, one after the other, and don't answer questions unless a harness is stuck. Note the minutes. You should see: two finished runs per task, each in its own folder.

Step 8 · Choose tools (10 minutes). Score each run: known answer passed or not, minutes, and yes or no for each habit: showed the fault first, checked its own work, final message matched what it did. Put everything in one table, ten rows. You should see: a table with ten rows and no empty cells.

Exit check

Hand in the ten-row table and a recommendation of at most five lines: the harness you pick, the score that decided it, the one habit that mattered most, what your five tasks did not cover, and when to test again.

It passes when a colleague could rerun one row from your notes and get the same result, every known answer lives outside the project folders, and the recommendation names at least one limit of the test.

Rule to remember

Test harnesses on your own tasks, one fresh copy per run, and trust only the answers you wrote down before the run.

Details for experts

Checked on 30 September 2026. You don't need this to follow the book; it's for setting things up or answering a colleague.

Further reading and tools

• T20-L01 · Same Model, Different Harness: See What the Program Around the AI Changes
• T08-L02 · Get a Script Your Colleague Can Rerun, Without Being a Developer
• T16-L03 · Test a Tool on Your Own Work Before Your Team Depends on It
• DeepSeek Harness: Taught
• OpenCode: Taught
• OpenHands: Catalogued