Spec-driven development

At Level 3 Builder, another person may depend on code you direct an agent to change. The goal is no longer merely to get code that runs. You need a change whose intended behavior was agreed before implementation, whose boundaries are visible in the diff, and whose evidence lets a reviewer accept or reject it without trusting the agent's summary.

2. Nine files for a one-sentence request

You ask a coding agent to "validate the signal before calculating the summary." It returns 600 lines across nine files. It adds a validation framework, changes error messages, reformats unrelated functions, introduces a package, and rewrites tests. Somewhere inside that patch may be the right check. Nobody can tell in ten minutes whether existing behavior survived.

The Company version looks equally reasonable. You ask for one reference field on a fictional internal ticket. The patch renames the ticket model, changes the stored schema, adds search, and updates every caller. The new field appears on screen, so the demonstration passes. Yet the patch also turns a display-only request into a migration and changes title formatting.

The failure began before code was generated. "Validate the signal" and "add a reference field" name directions, not reviewable outcomes. They do not say which inputs pass, which fail, what a failure changes, or what existing behavior is protected. The agent filled those gaps with plausible decisions. A large diff then hid those decisions from the reviewer.

3. After this you can

• Write a narrow specification with observable outcomes and ordinary, boundary, and failure criteria.
• Protect files, interfaces, data effects, dependencies, and existing behavior with explicit scope and non-goals.
• Constrain an agent to a diff a human can review in about ten minutes.
• Test and review an implementation against requirement identifiers.
• Reject or revise with a specific, reproducible reason when code fails the contract or exposes a wrong decision.

4. Prerequisites

• T08-L02 · Code without being a developer, especially tracing inputs, validation, transformation, outputs, and the entry point.
• T03-L02 · Context engineering, so the agent receives the current, minimal, authoritative context rather than a repository dump.
• Python 3, a text editor, and a disposable folder or branch containing only synthetic exercise code.
• An organisation-approved coding assistant with a plan or read-only mode and a complete diff view.
• About 90 minutes for the independent exercise.

Use only public, synthetic, or explicitly approved code and data. Do not place credentials, .env files, customer records, participant or patient data, unpublished results, production configuration, or private repository content into an unapproved assistant. A written specification limits the task; it does not technically enforce access. Open only the disposable project, deny unnecessary network and shell access, review every command, and do not grant deployment, merge, database, or production authority.

If the requested change affects safety, regulated data, permissions, money, employment, or scientific validity, an accountable domain owner must define and approve the governing rule. Stop when that rule is absent. A coding agent should not invent it, and a passing unit test cannot make an invented rule legitimate.

5. The idea in one page

Spec-driven development reverses the order: write observable requirements and acceptance criteria; state what may and must not change; review a plan mapped to those statements; permit a small implementation; run the named checks and inspect the diff; then accept or reject by citing the specification. An agent may propose code, tests, or wording. The accountable human still owns product intent, unresolved assumptions, and release.

The narrow change contract

A useful specification is a small contract between the requester, implementer, and reviewer. It answers three questions:

What changes?
What must not change?
How will we verify both?

Use this compact structure:

Change ID and owner
Problem and observable outcome
Current behavior
Required behavior, with numbered acceptance criteria
Allowed files and interfaces
Must not change / non-goals
Assumptions and unresolved decisions
Verification matrix: criterion -> check -> expected evidence
Stop conditions

The problem explains why the change matters. The required behavior states what a caller or user can observe. The allowed scope bounds the proposed diff. The must not change list protects behavior that is easy to damage accidentally. The verification matrix says what evidence will decide acceptance. A stop condition turns uncertainty or scope expansion into a pause rather than an improvisation.

Specify outcomes, not your first solution

Add a regex in analysis.py describes a mechanism. It may force the wrong design and gives no expected behavior. Prefer:

When any supplied signal is blank, non-numeric, non-finite, or negative, summary creation
stops with ValueError identifying the 1-based data-row number. No summary is returned.

That statement permits several implementations while giving tests and reviewers something observable. Implementation constraints belong in the specification only when they are genuinely binding. No new dependency can be legitimate when deployment policy or the small change boundary requires it. Use a class called SignalValidator is normally a design guess for the agent's plan, not product behavior.

Avoid words such as improve, robust, clean, properly, support, and user-friendly unless a measurable condition follows. "Handle invalid references properly" invites invention. "After trimming outer whitespace, reject a reference longer than 20 characters with ValueError; return no payload" is testable.

Acceptance criteria cover behavior and preservation

Write criteria before the agent writes code. Give each an identifier so plans, tests, and review comments can point to the same rule. Cover at least:

"It runs" proves only that one path did not crash. It does not prove the boundary, denied behavior, lack of side effects, or preservation of existing behavior. A test name is also not proof by itself; inspect whether its input and assertion actually represent the criterion.

6. The worked example: two narrow changes

Both lanes use synthetic data and the Python standard library. Choose one lane when practising. The examples show the complete loop: contract, bad attempt, evidence-based rejection, corrected implementation, tests, and review.

Keep the plan and review bounded

Ten minutes is a scope test, not a universal review deadline. Name the allowed files, permit one behavior, and exclude drive-by formatting, unrelated renames, and dependencies. Security-critical or unfamiliar code may need longer; if an informed reviewer cannot map the diff to the criteria quickly, split the change.

Ask for a plan before edits:

Read SPEC.md as the authorized change contract. Do not edit or run commands yet.
Return a table: proposed change, file, acceptance criterion, verification.
List assumptions. Omit work that cannot be mapped. Stop on contradiction or scope expansion.

After implementation, inspect allowed paths, map each changed block and test to a criterion, check protected behavior, retain exact command output, and mark uncertainty. Reject with the criterion, observed mismatch, reproduction, and smallest next step. If implementation exposes a wrong requirement, pause for the owner, version the changed criterion, and rerun the complete matrix.

Lab lane: reject invalid synthetic signals

The fictional analysis.py already summarizes signal strings:

from statistics import mean

def summarize(rows):
    values = [float(row["signal"]) for row in rows]
    return {"count": len(values), "mean_signal": round(mean(values), 2)}

Its existing known behavior is [{"signal": "10"}, {"signal": "14"}] returning {"count": 2, "mean_signal": 12.0}. The accountable synthetic exercise owner decides that blank, non-numeric, non-finite, and negative values must stop the whole summary. This is an exercise rule, not a general scientific rule.

Lab change contract

ID: LAB-SIGNAL-VALIDATION-1
Owner: synthetic exercise owner

Problem
Invalid signal text currently fails inconsistently; NaN and negative values can reach the mean.

Outcome
Reject an invalid signal before calculating and return the existing summary unchanged for valid rows.

Acceptance criteria
AC-L1: Two valid rows "10" and "14" return exactly
       {"count": 2, "mean_signal": 12.0}.
AC-L2: Blank or non-numeric signal raises ValueError containing the 1-based data-row number.
AC-L3: Negative, NaN, or Infinity signal raises ValueError containing that row number.
AC-L4: On any invalid row, summarize returns no partial result.

Allowed change
analysis.py and test_analysis.py only. Python standard library only.

Must not change / non-goals
NC-L1: Do not change the summarize(rows) interface, valid-result keys, rounding, row order,
file I/O, dependencies, other functions, or repository configuration.
NC-L2: Do not skip or repair invalid rows.

Assumptions
rows is an existing iterable of mappings. Empty-input behavior is outside this change.

Verification
AC-L1 -> exact equality unit test.
AC-L2 -> blank and "missing" tests assert ValueError and row number.
AC-L3 -> -1, NaN, and Infinity tests assert ValueError and row number.
AC-L4 -> all failure tests assert that the call raises instead of returning.
Scope -> changed paths are exactly analysis.py and test_analysis.py; diff check is clean.

Stop
Stop if negative values are scientifically valid, the interface must change, another source file
is needed, or current tests contradict this contract. Ask the exercise owner before editing.

The plan should map one validation block in analysis.py and focused cases in test_analysis.py to AC-L1 through AC-L4. It should not propose a framework or a new parser module.

Rejected Lab attempt

Suppose the first attempt uses this comprehension:

values = [float(row["signal"]) for row in rows if float(row["signal"]) >= 0]

It runs for valid inputs. It also silently drops -1, evaluates each accepted value twice, lets NaN through, and cannot identify the rejected row. Reject it against the contract:

REJECT AC-L3 and NC-L2. For [{"signal": "10"}, {"signal": "-1"}], the patch returns
{"count": 1, "mean_signal": 10.0}. AC-L3 requires ValueError naming row 2, and the
non-goal says not to skip invalid rows. Replace only the validation logic and focused tests.

This is not a style dispute about comprehensions. The attempt violates observable failure and preservation rules.

Corrected Lab implementation

import math
from statistics import mean

def summarize(rows):
    values = []
    for row_number, row in enumerate(rows, start=1):
        try:
            value = float(row["signal"])
        except (KeyError, TypeError, ValueError) as error:
            raise ValueError(f"Data row {row_number}: invalid signal") from error
        if not math.isfinite(value) or value < 0:
            raise ValueError(f"Data row {row_number}: invalid signal")
        values.append(value)
    return {"count": len(values), "mean_signal": round(mean(values), 2)}

test_analysis.py supplies focused evidence:

import unittest

from analysis import summarize

class SummarizeTests(unittest.TestCase):
    def test_ac_l1_preserves_valid_summary(self):
        rows = [{"signal": "10"}, {"signal": "14"}]
        self.assertEqual(summarize(rows), {"count": 2, "mean_signal": 12.0})

    def test_ac_l2_rejects_bad_text_with_row_number(self):
        for bad_signal in ("", "missing"):
            with self.subTest(signal=bad_signal):
                with self.assertRaisesRegex(ValueError, "Data row 2"):
                    summarize([{"signal": "10"}, {"signal": bad_signal}])

    def test_ac_l3_rejects_out_of_range_values(self):
        for bad_signal in ("-1", "NaN", "Infinity"):
            with self.subTest(signal=bad_signal):
                with self.assertRaisesRegex(ValueError, "Data row 2"):
                    summarize([{"signal": "10"}, {"signal": bad_signal}])

if __name__ == "__main__":
    unittest.main()

Run python -m unittest -v. Confirm three test methods pass, including every subtest. Then inspect the changed-file list and complete diff. AC-L4 is demonstrated because each invalid call must raise; no assertion accepts a shortened summary. The existing interface, output keys, and rounding line remain unchanged. Record that empty-input behavior was not tested because the contract explicitly excluded it; do not imply broader coverage.

A concise review record is:

LAB-SIGNAL-VALIDATION-1 REVIEW
Paths: analysis.py, test_analysis.py only - PASS
AC-L1 valid exact output - PASS
AC-L2 blank/non-numeric with row number - PASS
AC-L3 negative/non-finite with row number - PASS
AC-L4 no partial return - PASS
Non-goals: interface, keys, rounding, dependencies unchanged - PASS
Command: python -m unittest -v - PASS (3 methods, 5 subtests)
Rejected attempt retained: silent negative-row filtering violated AC-L3 and NC-L2
Limit: synthetic exercise rule; no claim about real instrument validity
Decision: ACCEPT for the exercise

Company lane: add one reference field

The fictional internal tool creates a display payload in tickets.py:

def ticket_payload(title, owner):
    return {"title": title.strip(), "owner": owner}

The request is to add an optional external reference for display. There is no database, API, authentication, or production service in this exercise.

Company change contract

ID: COMPANY-TICKET-REFERENCE-1
Owner: synthetic tool owner

Problem
Reviewers cannot display the fictional source reference beside a ticket.

Outcome
Callers may supply an optional reference, and the returned payload displays its normalized value.

Acceptance criteria
AC-C1: ticket_payload(" Printer ", "Sam", " REF-204 ") returns exactly
       {"title": "Printer", "owner": "Sam", "reference": "REF-204"}.
AC-C2: Omitting reference returns the same title and owner values plus "reference": "".
AC-C3: After trimming outer whitespace, a 20-character reference succeeds; a 21-character
       reference raises ValueError.
AC-C4: A non-string reference raises TypeError; no payload is returned.

Allowed change
tickets.py and test_tickets.py only. The optional third function parameter may be added.

Must not change / non-goals
NC-C1: Do not change title or owner normalization or the existing parameter order.
NC-C2: Do not change authorization, persistence, API schemas, search, UI layout, dependencies,
logging, or any other file. Do not invent a reference.

Assumptions
Existing callers use two positional arguments. The field is display-only synthetic text.

Verification
AC-C1 -> exact three-field payload test.
AC-C2 -> existing two-argument call with exact payload test.
AC-C3 -> padded inputs that trim to length 20 and 21 prove trim-before-length behavior;
         the first returns the exact 20 characters and the second raises ValueError.
AC-C4 -> integer reference TypeError test.
Scope -> exactly tickets.py and test_tickets.py changed; diff check is clean.

Stop
Stop if the field must be stored, searched, sent over an API, restricted by a real policy, or
added through changes to other callers. Those are separate decisions and specifications.

Rejected Company attempt

Imagine that the agent adds the field but also "standardizes" titles:

def ticket_payload(title, owner, reference=""):
    return {
        "title": title.strip().title(),
        "owner": owner,
        "reference": str(reference).strip()[:20],
    }

The demonstration with " Printer " looks correct. The implementation still fails the contract: it changes existing title capitalization, converts an integer instead of rejecting it, and silently truncates 21 characters. The review should say:

REJECT AC-C3, AC-C4, and NC-C1. A 21-character reference is truncated rather than rejected;
reference=204 becomes "204" rather than raising TypeError; and title "API error" changes to
"Api Error" although title behavior is protected. Reproduce with those three inputs. Restore
the existing title expression and implement only the specified reference behavior.

The rejection does not argue whether title case is attractive. It proves three contract violations.

Corrected Company implementation

def ticket_payload(title, owner, reference=""):
    if not isinstance(reference, str):
        raise TypeError("reference must be a string")
    normalized_reference = reference.strip()
    if len(normalized_reference) > 20:
        raise ValueError("reference must be at most 20 characters")
    return {
        "title": title.strip(),
        "owner": owner,
        "reference": normalized_reference,
    }

test_tickets.py:

import unittest

from tickets import ticket_payload

class TicketPayloadTests(unittest.TestCase):
    def test_ac_c1_adds_trimmed_reference(self):
        actual = ticket_payload(" Printer ", "Sam", " REF-204 ")
        expected = {"title": "Printer", "owner": "Sam", "reference": "REF-204"}
        self.assertEqual(actual, expected)

    def test_ac_c2_keeps_two_argument_calls(self):
        actual = ticket_payload("API error", "Lee")
        expected = {"title": "API error", "owner": "Lee", "reference": ""}
        self.assertEqual(actual, expected)

    def test_ac_c3_checks_trimmed_boundary(self):
        twenty_characters = "R" * 20
        self.assertEqual(
            ticket_payload("T", "Lee", f"  {twenty_characters}  ")["reference"],
            twenty_characters,
        )
        with self.assertRaises(ValueError):
            ticket_payload("T", "Lee", f"  {'R' * 21}  ")

    def test_ac_c4_rejects_non_string_reference(self):
        with self.assertRaises(TypeError):
            ticket_payload("T", "Lee", 204)

if __name__ == "__main__":
    unittest.main()

Run python -m unittest -v, inspect both changed paths, and compare each assertion with its criterion. The two-argument test protects existing callers and title capitalization. The padded boundary inputs prove that outer whitespace is removed before the 20-versus-21 length decision: the value that trims to 20 is returned exactly, while the value that trims to 21 raises ValueError. No test or code claims to persist, authorize, search, or transmit the field.

Record the result in the same review format as the Lab lane. Keep the rejected attempt and reason beside the accepted diff. That rejected evidence shows that review was capable of disagreeing with a plausible implementation; it is not a request to preserve unsafe code in an executable file.

When the specification changes

Suppose review reveals that Company references may contain 24 characters. The corrected code is not permission to keep the 20-character rule. Mark AC-C3 unresolved, ask the synthetic tool owner, and stop. If the owner confirms 24, revise the criterion and its boundary tests together, record 20 -> 24 and the reason, then ask for a new implementation. Rerun all tests, including title preservation and non-string failure. Never edit only the failing assertion to make existing code green.

In the Lab lane, discovering valid negative calibrated signals is even more important. The exercise owner or domain specialist must decide the valid range and units. Until then, neither the original requirement nor the implementation is acceptable for real analysis.

7. What goes wrong

The specification describes the solution

Symptom: the contract says create SignalValidator with regex X but never states which signals pass, fail, or remain unchanged.

Fix: move proposed classes, functions, and algorithms into the plan. Rewrite requirements as observable inputs, outputs, errors, and preserved behavior. Keep a technical constraint only when an authoritative source makes it binding.

There is no must-not-change clause

Symptom: the requested field appears, but title formatting, dependencies, and storage also change.

Fix: list protected interfaces, outputs, paths, data effects, dependencies, and operations. Add preservation tests for the most likely regression. Say what must not change even when it feels obvious.

The change is too large to review

Symptom: one behavior arrives with refactors, renames, generated files, formatting, and dependency updates.

Fix: reject unmapped work. Separate prerequisite refactoring into its own justified change, or choose a smaller implementation. Ask for no drive-by cleanup and inspect the changed-path list before reading details.

Acceptance means only that it runs

Symptom: the happy-path demonstration succeeds, so failures, boundaries, and side effects are assumed correct.

Fix: map each criterion to an independent check. Test exact output, boundary values, denied input, preservation, and scope. Retain command output and inspect what each test actually asserts.

Review becomes a taste argument

Symptom: comments say "too complicated" or "I would write this differently," and the agent responds with another unrelated design.

Fix: cite the criterion or non-goal, provide an observed mismatch and reproduction, then request the smallest correction. If the implementation satisfies the contract and the remaining concern is important, revise the contract rather than moving an invisible goalpost.

The specification is never updated

Symptom: implementation exposes a wrong limit, but code and tests quietly adopt a new number while the contract retains the old one.

Fix: pause and obtain the accountable decision. Version the criterion, record why it changed, update expected evidence, and rerun the complete verification matrix. History should show whether intent changed or implementation failed.

Passing tests are generated to match the bug

Symptom: the agent changes implementation and expected values together, so every test passes while protected behavior disappears.

Fix: write expected outcomes before implementation, review test diffs independently, and retain a known-behavior test. A green test suite is useful only when its assertions still represent authorized intent.

The contract is mistaken for enforcement

Symptom: broad filesystem, network, or deployment access is granted because the prompt says not to use it.

Fix: combine written boundaries with a disposable working set, least-privilege tool permissions, explicit command approval, and human review. A non-goal documents a violation; technical controls reduce the opportunity for one.

8. Do it yourself: one accepted change in 90 minutes

Choose either the Lab or Company lane. Use the supplied synthetic code or an equally small public, synthetic, or explicitly approved example. Your final submission is one change record, not two separate deliverables.

Minutes 0-10: reproduce the current behavior and record the exact command and output. Name one owner and one problem. Confirm that the folder contains no secrets or unrelated private material. If the current behavior is unknown, establish it before specifying a change.

Minutes 10-25: write the contract: observable outcome; three to five numbered acceptance criteria; allowed files; protected behavior and non-goals; assumptions; verification matrix; and stop conditions. Include ordinary, boundary, failure, preservation, and changed-path evidence. Ask the owner to resolve any domain or policy ambiguity.

Minutes 25-35: ask the approved coding agent for a plan only. Require every action to map to a criterion. Reject a plan that adds a dependency, touches an unnamed path, combines cleanup, invents a product rule, or lacks a failure test. Save the accepted plan in the change record.

Minutes 35-50: allow edits only to the named files. Ask for the complete changed-path list and diff. Do not approve installation, network, merge, deployment, database, or production commands. If scope expands, stop rather than editing the contract merely to excuse the patch.

Minutes 50-60: inspect the first attempt. Deliberately test one boundary or failure likely to expose a mismatch. If the attempt fully passes, create a safe deliberately nonconforming candidate in prose, such as silent truncation, and review that proposal without placing it in executable code. Record one rejection citing the criterion, observation, reproduction, and required correction.

Minutes 60-72: ask for the smallest corrected implementation. Review tests separately from implementation: ensure expected values came from the contract, not from the current output. Check every changed block maps to a criterion and every changed path is allowed.

Minutes 72-82: run the exact verification commands. Retain the command, result, test count, and any unrun check. Inspect valid output, boundary behavior, the failure message, protected behavior, changed paths, and a whitespace or diff check available in your environment.

Minutes 82-90: complete the review decision. If implementation revealed a wrong requirement, do not hide it: record the owner decision, revise the criterion, and rerun affected and preservation tests. Package the specification, accepted implementation diff, test evidence, review checklist, and rejected attempt with its reason into one change record.

Stop without accepting if you cannot identify the owner, expected result, permitted data, exact changed paths, or evidence for a consequential rule. An honest unresolved record is safer than a fabricated pass.

9. Exit check

Deliver exactly one artifact: one reviewable change record containing the specification, final implementation, and one rejected attempt with its reason.

It passes when another person can find the owner and problem; trace every final changed block and test to a numbered criterion; reproduce the ordinary, boundary, failure, preservation, and scope checks; see that only allowed files changed; understand what was deliberately excluded; and read one rejection that cites a concrete contract violation. Include exact commands and actual results, and mark checks that were not run. If the specification changed, include the old rule, accountable decision, reason, new rule, and rerun evidence in the same record.

The specification, accepted diff, tests, review result, and rejected-attempt note are components of one artifact. Do not submit real sensitive data, credentials, production access, a deployed system, or an unreviewed agent transcript.

10. Rule to remember

Say what must not change.

11. Further reading & tools

• Taught: T08-L02 · Code without being a developer - establishes the bounded script, code-reading landmarks, known-input check, and repair loop this book turns into a team-reviewable change.
• Taught: T03-L02 · Context engineering - supplies only the current specification and relevant code context while excluding stale or unauthorized material.
• Taught: Product requirements & spec engineering - defines observable requirements, scope boundaries, action and failure rules, and a verification matrix.
• Taught: Write a narrow change contract - practises one-file scope, exclusions, evidence, and stop conditions before editing.
• Catalogued · Primary: NIST SP 800-218, Secure Software Development Framework (opens in a new tab) - primary guidance for defining security requirements and retaining evidence that software satisfies them.
• Catalogued · Primary: OWASP Code Review Guide (opens in a new tab) - systematic code-review guidance; use it proportionately for the risk and scope of the change.
• Catalogued · Primary: Python unittest documentation (opens in a new tab) - primary reference for the standard-library test structures used in both synthetic implementations.
• Catalogued: Claude Code and Codex - agent interfaces that can support plan, edit, diff, and test workflows when approved and permission-bounded; product controls may change.
• Catalogued: T08-L04 · Working in a real repo - continues from one reviewable local change to branches, pull requests, required checks, and shared-repository protection.
• Catalogued: Tools index - compare tools only after the owner, specification, scope, data policy, permissions, and acceptance evidence are defined.