T10-L03

Data & analysis · Builder

A reproducible analysis pipeline

At Level 3 Builder, colleagues depend on something you built. A correct figure on your computer is not enough: raw evidence, transformation steps, parameters, checks, and run instructions must let another person regenerate it without your memory or hidden setup.

Level
BuilderLevel 3 of 5
Curriculum position
Family 2 · Track 10
Reading time
90 minutes
Reading progress
0%Time on this book
Last revised
Sep 5, 2026

At Level 3 Builder, colleagues depend on something you built. A correct figure on your computer is not enough: raw evidence, transformation steps, parameters, checks, and run instructions must let another person regenerate it without your memory or hidden setup.

2. The number nobody can rebuild

You open a report before a review meeting and find a chart labelled "Mean value by group." A colleague asks which rows produced the taller bar. The analyst who made it is on leave. Their shared folder contains final.csv, final-fixed.csv, a screenshot, and a script that refers to C:\Users\name\Desktop\August. Nobody knows whether the screenshot came before or after the manual fix.

The number may be right, but the team cannot demonstrate that. Re-running the script fails on another computer. Replacing its path makes it run, yet the result differs because one parameter lived in an old notebook cell. In a lab, this blocks review of a result. In a company, it delays a monthly report and forces someone to reverse-engineer undocumented choices.

You need a pipeline that takes a colleague from a clean checkout to the same checked table and figure: immutable raw input, versioned steps, central parameters, regenerated output, tests, and a self-contained README.

3. After this you can

  • Separate raw data, analysis steps, and generated output so edits cannot be mistaken for computation.
  • Centralise paths, labels, and acceptance values in one explicit parameter file.
  • Build a deterministic analysis that produces the same table and figure from the same recorded inputs.
  • Test a clean-room run, raw-file preservation, deliberate failure, and byte-identical rerun.
  • Hand one versioned pipeline commit to a colleague and collect reproducibility evidence without coaching the run.

4. Prerequisites

  • T10-L02 - Clean a messy export, including preservation of raw input and explicit transformations.
  • T08-L02 - Code without being a developer, including bounded code review, known-input checks, and deliberate failures.
  • Python 3 with the standard library, Git, and a plain-text editor or approved coding assistant.
  • Permission to create a local Git repository and run a script in a disposable exercise folder.
  • A colleague who can perform the final handoff from written instructions alone.

Use only public, synthetic, course-provided, or explicitly approved data. Both fixtures below are synthetic. Do not put patient, participant, employee, customer, credential, unpublished, or production data into an exercise repository. Git retains history: deleting a sensitive file in a later commit does not remove it from earlier commits. If unsuitable data enters history, stop sharing the repository and follow your organisation's incident and history-remediation process.

5. The idea in one page

Give every kind of file one job

Use three top-level folders and do not blur their roles:

FolderContainsRule
raw/The received input used for this runNever edit it in place. Replace it only through a documented new input or version.
steps/Code or an exported visual workflow, plus automated testsVersion it. Every transformation must be visible here.
output/Tables, figures, and other derived filesRegenerate it. Never correct it by hand.

Keep parameters.json and README.md at the project root. The parameter file is the one place for input and output paths, column choices, labels, group order, expected row count, and the hand-calculated acceptance result. The README states the environment, exact commands, generated files, expected evidence, and limits. Git identifies the complete version a colleague tested.

This separation creates a useful test: delete output/, follow the README, and see whether the pipeline reconstructs it from raw/, steps/, and parameters.json. If deletion destroys information needed for the analysis, hidden work was living in the output folder.

Define "the same result"

Do not settle for "the bars look similar." For this bounded pipeline, the same result means:

  1. The CSV has the exact group order, counts, totals, and means declared before the run.
  2. The SVG is generated from that same summary, not edited separately.
  3. The raw input hash is unchanged by execution.
  4. A second run with the same commit and parameters produces byte-identical CSV and SVG files.
  5. A malformed input stops before any plausible replacement output is written.

Byte identity is practical here because the outputs contain no timestamps, random values, machine paths, locale-dependent formatting, or unstable row order. Other analyses may legitimately produce equivalent rather than byte-identical files, but then the README must define the comparison in advance, such as a numeric tolerance, sorted row set, or image-data checksum. Never weaken the comparison only after observing a mismatch.

Remove hidden state

Hidden state is anything the pipeline needs but the project does not declare: an absolute path, an unexecuted notebook cell, a package installed months ago, a manually selected spreadsheet range, an environment variable, a locale-specific decimal rule, or an output left from the previous run. A successful run on the author's machine does not reveal these dependencies.

The example uses only the Python standard library, resolves paths relative to parameters.json, orders groups and decimals explicitly, recreates outputs, and tests in a temporary directory without old output. These behaviours make a clean handoff possible.

Treat reproducibility as an independent test

The builder runs the tests first, but the exit check belongs to a colleague. Give them a commit identifier and the README, then stay silent unless the instructions themselves tell them to ask for a documented prerequisite. They should start from a fresh clone or exported copy, run the tests, remove generated output, run the analysis, compare the declared values and hashes, and record the result.

If they need an oral instruction, the handoff fails. Add the missing instruction to the README, commit it, and ask them to restart from the new commit. Do not quietly coach them through the old version and call it reproducible.

6. The worked example: rebuild one result from a clean checkout

The Lab and Company framings use the same project shape, script, tests, commands, and evidence. Only the synthetic rows, labels, and declared acceptance result change. Choose one framing; do not combine both fixtures in one run.

Assemble the shared project

Create a folder named reproducible-analysis with this structure:

reproducible-analysis/
|-- .gitignore
|-- README.md
|-- parameters.json
|-- raw/
|   `-- input.csv
|-- steps/
|   |-- analyze.py
|   `-- test_pipeline.py
`-- output/                 # generated; may be absent before a run

Create .gitignore before running Python so interpreter artifacts cannot enter the handoff:

__pycache__/
*.py[cod]
.venv/

Do not ignore output/: its visible untracked or changed state helps the runner confirm what this execution generated. The code below rejects absolute paths, paths that escape the project, an input outside raw/, and outputs outside output/. Save it as steps/analyze.py:

import argparse
import csv
import io
import json
import os
import shutil
import tempfile
from decimal import Decimal, InvalidOperation
from html import escape
from pathlib import Path

def project_path(root, value, required_folder):
    if not isinstance(value, str) or not value:
        raise ValueError("Paths must be non-empty strings")
    if Path(value).is_absolute():
        raise ValueError(f"Path must be relative: {value}")
    project_root = root.resolve()
    candidate = (root / value).resolve()
    boundary = (root / required_folder).resolve()
    try:
        boundary.relative_to(project_root)
        candidate.relative_to(boundary)
    except ValueError as error:
        raise ValueError(f"Path must be inside {required_folder}/: {value}") from error
    return candidate

def read_parameters(path):
    with path.open(encoding="utf-8") as source:
        params = json.load(source)
    if not isinstance(params, dict):
        raise ValueError("parameters.json must contain one JSON object")
    required = {
        "input", "output_csv", "output_svg", "id_column", "group_column",
        "value_column", "group_order", "group_labels", "expected_row_count",
        "expected_summary",
    }
    missing = required - set(params)
    if missing:
        raise ValueError(f"Missing parameters: {', '.join(sorted(missing))}")
    group_order = params["group_order"]
    labels = params["group_labels"]
    if (
        not isinstance(group_order, list)
        or not group_order
        or any(not isinstance(group, str) or not group for group in group_order)
        or len(group_order) != len(set(group_order))
    ):
        raise ValueError("group_order must contain unique non-empty strings")
    if not isinstance(labels, dict) or set(labels) != set(group_order):
        raise ValueError("group_labels must contain exactly one label per group")
    if any(not isinstance(label, str) or not label for label in labels.values()):
        raise ValueError("group labels must be non-empty strings")
    if not isinstance(params["expected_row_count"], int) or params["expected_row_count"] < 1:
        raise ValueError("expected_row_count must be a positive integer")
    if not isinstance(params["expected_summary"], list):
        raise ValueError("expected_summary must be a list")
    return params

def read_rows(input_path, params):
    with input_path.open(newline="", encoding="utf-8") as source:
        reader = csv.DictReader(source, strict=True)
        required = {
            params["id_column"], params["group_column"], params["value_column"]
        }
        missing = required - set(reader.fieldnames or [])
        if missing:
            raise ValueError(f"Missing columns: {', '.join(sorted(missing))}")

        rows = []
        seen_ids = set()
        allowed_groups = set(params["group_order"])
        for line_number, row in enumerate(reader, start=2):
            if None in row:
                raise ValueError(
                    f"Line {line_number}: malformed CSV or extra fields"
                )
            record_id = (row[params["id_column"]] or "").strip()
            group = " ".join((row[params["group_column"]] or "").split()).casefold()
            value_text = (row[params["value_column"]] or "").strip()
            if not record_id or not group or not value_text:
                raise ValueError(f"Line {line_number}: blank required value")
            if record_id in seen_ids:
                raise ValueError(f"Line {line_number}: duplicate ID {record_id}")
            if group not in allowed_groups:
                raise ValueError(f"Line {line_number}: unexpected group {group}")
            try:
                value = Decimal(value_text)
            except InvalidOperation as error:
                raise ValueError(f"Line {line_number}: invalid number {value_text}") from error
            if not value.is_finite():
                raise ValueError(f"Line {line_number}: number must be finite")
            seen_ids.add(record_id)
            rows.append((group, value))

    if len(rows) != params["expected_row_count"]:
        raise ValueError(
            f"Expected {params['expected_row_count']} rows, received {len(rows)}"
        )
    return rows

def summarize(rows, group_order):
    counts = {group: 0 for group in group_order}
    totals = {group: Decimal("0") for group in group_order}
    for group, value in rows:
        counts[group] += 1
        totals[group] += value

    summary = []
    for group in group_order:
        if counts[group] == 0:
            raise ValueError(f"Group has no rows: {group}")
        total = totals[group]
        mean = total / counts[group]
        summary.append({
            "group": group,
            "count": counts[group],
            "total": f"{total:.2f}",
            "mean": f"{mean:.2f}",
        })
    return summary

def verify_expected(summary, expected):
    if summary != expected:
        raise ValueError(f"Summary differs from declared acceptance result: {summary}")

def render_csv(summary):
    target = io.StringIO(newline="")
    writer = csv.DictWriter(
        target,
        fieldnames=["group", "count", "total", "mean"],
        lineterminator="\n",
    )
    writer.writeheader()
    writer.writerows(summary)
    return target.getvalue()

def render_svg(summary, labels):
    means = [Decimal(row["mean"]) for row in summary]
    if any(mean < 0 for mean in means) or max(means) == 0:
        raise ValueError("SVG requires non-negative means and at least one positive mean")
    maximum = max(means)
    bars = []
    for index, row in enumerate(summary):
        y = 28 + index * 48
        width = int(Decimal(row["mean"]) / maximum * 360)
        label = escape(labels[row["group"]])
        text = escape(f"{label}: mean {row['mean']} (n={row['count']})")
        bars.append(f'<text x="10" y="{y + 16}">{text}</text>')
        bars.append(
            f'<rect x="220" y="{y}" width="{width}" height="24" fill="#315f72"/>'
        )
    height = 52 + len(summary) * 48
    svg = (
        f'<svg xmlns="http://www.w3.org/2000/svg" width="620" height="{height}" '
        'role="img" aria-labelledby="figure-title figure-description">\n'
        '<title id="figure-title">Mean value by group</title>\n'
        '<desc id="figure-description">Horizontal bars compare the mean value '
        'for each group; labels also give each mean and sample count.</desc>\n'
        '<style>text { font: 14px sans-serif; }</style>\n'
        + "\n".join(bars)
        + "\n</svg>\n"
    )
    return svg

def write_outputs(csv_text, svg_text, csv_path, svg_path, output_root):
    staging_root = Path(tempfile.mkdtemp(dir=output_root.parent, prefix=".output-stage-"))
    backup_root = Path(tempfile.mkdtemp(dir=output_root.parent, prefix=".output-backup-"))
    backup_root.rmdir()
    had_output = output_root.exists()
    try:
        staged_csv = staging_root / csv_path.relative_to(output_root)
        staged_svg = staging_root / svg_path.relative_to(output_root)
        for path, text in ((staged_csv, csv_text), (staged_svg, svg_text)):
            path.parent.mkdir(parents=True, exist_ok=True)
            with path.open("w", newline="\n", encoding="utf-8") as target:
                target.write(text)

        if had_output:
            os.replace(output_root, backup_root)
        try:
            os.replace(staging_root, output_root)
        except OSError:
            if had_output and backup_root.exists():
                os.replace(backup_root, output_root)
            raise
        if had_output:
            shutil.rmtree(backup_root)
    finally:
        if staging_root.exists():
            shutil.rmtree(staging_root)

def run(parameter_path):
    parameter_path = parameter_path.resolve()
    root = parameter_path.parent
    params = read_parameters(parameter_path)
    input_path = project_path(root, params["input"], "raw")
    csv_path = project_path(root, params["output_csv"], "output")
    svg_path = project_path(root, params["output_svg"], "output")
    output_root = (root / "output").resolve()
    if csv_path == svg_path:
        raise ValueError("CSV and SVG output paths must differ")
    rows = read_rows(input_path, params)
    summary = summarize(rows, params["group_order"])
    verify_expected(summary, params["expected_summary"])
    csv_text = render_csv(summary)
    svg_text = render_svg(summary, params["group_labels"])
    write_outputs(csv_text, svg_text, csv_path, svg_path, output_root)
    print(f"PASS: {len(rows)} rows, {len(summary)} groups, outputs regenerated")

if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument("--parameters", type=Path, required=True)
    arguments = parser.parse_args()
    try:
        run(arguments.parameters)
    except (
        ArithmeticError, KeyError, OSError, TypeError, ValueError,
        csv.Error, json.JSONDecodeError,
    ) as error:
        raise SystemExit(f"FAIL: {error}") from error

The script completes every read, validation, calculation, and rendering step before staging a complete replacement output directory beside the current one. It moves the previous directory aside, atomically swaps the complete generation into place, and restores the previous directory if that swap fails. A parsing, acceptance, label, or chart-calculation failure therefore cannot leave a plausible new partial output. It constrains file roles and rejects a raw/ or output/ symlink that resolves outside the project, so a parameter cannot redirect generated content over the raw fixture. It does not choose a group implicitly, read an environment variable, call a network service, or depend on the current working directory. The declared group_order controls output order instead of relying on incidental input order.

Save this clean-room test as steps/test_pipeline.py:

import csv
import hashlib
import json
import shutil
import subprocess
import sys
import tempfile
import unittest
from pathlib import Path
from unittest.mock import patch

from steps import analyze

SOURCE_ROOT = Path(__file__).resolve().parents[1]

def digest(path):
    return hashlib.sha256(path.read_bytes()).hexdigest()

class PipelineTest(unittest.TestCase):
    def clean_copy(self, destination):
        shutil.copytree(SOURCE_ROOT / "raw", destination / "raw")
        (destination / "steps").mkdir()
        shutil.copy2(SOURCE_ROOT / "steps" / "analyze.py", destination / "steps")
        shutil.copy2(SOURCE_ROOT / "parameters.json", destination)

    def execute(self, root):
        return subprocess.run(
            [
                sys.executable,
                str(root / "steps" / "analyze.py"),
                "--parameters",
                str(root / "parameters.json"),
            ],
            cwd=root,
            capture_output=True,
            text=True,
        )

    def expected_csv(self, root):
        params = json.loads((root / "parameters.json").read_text(encoding="utf-8"))
        lines = ["group,count,total,mean"]
        for row in params["expected_summary"]:
            lines.append(f"{row['group']},{row['count']},{row['total']},{row['mean']}")
        return "\n".join(lines) + "\n"

    def update_parameters(self, root, change):
        path = root / "parameters.json"
        params = json.loads(path.read_text(encoding="utf-8"))
        change(params)
        path.write_text(json.dumps(params, indent=2) + "\n", encoding="utf-8")

    def test_clean_room_run_matches_contract_and_preserves_raw(self):
        with tempfile.TemporaryDirectory() as folder:
            root = Path(folder)
            self.clean_copy(root)
            raw_path = root / "raw" / "input.csv"
            before = digest(raw_path)
            result = self.execute(root)
            self.assertEqual(result.returncode, 0, result.stderr)
            self.assertEqual(before, digest(raw_path))
            self.assertEqual(
                (root / "output" / "summary.csv").read_text(encoding="utf-8"),
                self.expected_csv(root),
            )
            svg = (root / "output" / "figure.svg").read_text(encoding="utf-8")
            self.assertIn('role="img"', svg)
            self.assertIn('<title id="figure-title">', svg)
            self.assertIn('<desc id="figure-description">', svg)

    def test_second_run_is_byte_identical(self):
        with tempfile.TemporaryDirectory() as folder:
            root = Path(folder)
            self.clean_copy(root)
            first = self.execute(root)
            self.assertEqual(first.returncode, 0, first.stderr)
            first_hashes = [
                digest(root / "output" / "summary.csv"),
                digest(root / "output" / "figure.svg"),
            ]
            second = self.execute(root)
            self.assertEqual(second.returncode, 0, second.stderr)
            second_hashes = [
                digest(root / "output" / "summary.csv"),
                digest(root / "output" / "figure.svg"),
            ]
            self.assertEqual(first_hashes, second_hashes)

    def test_extra_csv_field_stops_before_output(self):
        with tempfile.TemporaryDirectory() as folder:
            root = Path(folder)
            self.clean_copy(root)
            input_path = root / "raw" / "input.csv"
            lines = input_path.read_text(encoding="utf-8").splitlines()
            lines[1] += ",unexpected"
            input_path.write_text("\n".join(lines) + "\n", encoding="utf-8")
            result = self.execute(root)
            self.assertNotEqual(result.returncode, 0)
            self.assertIn("malformed CSV or extra fields", result.stderr)
            self.assertFalse((root / "output").exists())

    def test_duplicate_stops_before_output(self):
        with tempfile.TemporaryDirectory() as folder:
            root = Path(folder)
            self.clean_copy(root)
            input_path = root / "raw" / "input.csv"
            with input_path.open(newline="", encoding="utf-8") as source:
                rows = list(csv.reader(source))
            with input_path.open("a", newline="", encoding="utf-8") as target:
                csv.writer(target, lineterminator="\n").writerow(rows[1])
            result = self.execute(root)
            self.assertNotEqual(result.returncode, 0)
            self.assertIn("duplicate ID", result.stderr)
            self.assertFalse((root / "output").exists())

    def test_missing_label_stops_before_output(self):
        with tempfile.TemporaryDirectory() as folder:
            root = Path(folder)
            self.clean_copy(root)
            self.update_parameters(
                root,
                lambda params: params["group_labels"].pop(params["group_order"][0]),
            )
            result = self.execute(root)
            self.assertNotEqual(result.returncode, 0)
            self.assertIn("exactly one label per group", result.stderr)
            self.assertFalse((root / "output").exists())

    def test_zero_chart_scale_stops_before_output(self):
        with tempfile.TemporaryDirectory() as folder:
            root = Path(folder)
            self.clean_copy(root)
            input_path = root / "raw" / "input.csv"
            with input_path.open(newline="", encoding="utf-8") as source:
                reader = csv.DictReader(source)
                rows = list(reader)
                fieldnames = reader.fieldnames
            for row in rows:
                row["value"] = "0"
            with input_path.open("w", newline="", encoding="utf-8") as target:
                writer = csv.DictWriter(target, fieldnames=fieldnames, lineterminator="\n")
                writer.writeheader()
                writer.writerows(rows)

            def expect_zero(params):
                for row in params["expected_summary"]:
                    row["total"] = "0.00"
                    row["mean"] = "0.00"

            self.update_parameters(root, expect_zero)
            result = self.execute(root)
            self.assertNotEqual(result.returncode, 0)
            self.assertIn("at least one positive mean", result.stderr)
            self.assertFalse((root / "output").exists())

    def test_output_cannot_overlap_raw_input(self):
        with tempfile.TemporaryDirectory() as folder:
            root = Path(folder)
            self.clean_copy(root)
            raw_path = root / "raw" / "input.csv"
            before = digest(raw_path)
            self.update_parameters(
                root,
                lambda params: params.update(output_csv="raw/input.csv"),
            )
            result = self.execute(root)
            self.assertNotEqual(result.returncode, 0)
            self.assertIn("inside output/", result.stderr)
            self.assertEqual(before, digest(raw_path))
            self.assertFalse((root / "output").exists())

    def test_failed_directory_swap_restores_previous_output(self):
        with tempfile.TemporaryDirectory() as folder:
            root = Path(folder)
            self.clean_copy(root)
            first = self.execute(root)
            self.assertEqual(first.returncode, 0, first.stderr)
            output_paths = [
                root / "output" / "summary.csv",
                root / "output" / "figure.svg",
            ]
            before = [digest(path) for path in output_paths]
            real_replace = analyze.os.replace
            replace_calls = 0

            def fail_second_replace(source, target):
                nonlocal replace_calls
                replace_calls += 1
                if replace_calls == 2:
                    raise OSError("injected directory swap failure")
                return real_replace(source, target)

            with patch.object(analyze.os, "replace", side_effect=fail_second_replace):
                with self.assertRaisesRegex(OSError, "injected directory swap failure"):
                    analyze.run(root / "parameters.json")

            self.assertEqual(before, [digest(path) for path in output_paths])
            self.assertFalse(list(root.glob(".output-*")))

if __name__ == "__main__":
    unittest.main()

Old output cannot make these tests pass: each uses a temporary project containing only the fixture, parameters, and script. They check declared values, raw preservation, repeatability, malformed CSV with an extra field, duplicate IDs, missing labels, invalid chart scale, path collision, and restoration after a failed directory swap.

Lab framing: a raw export becomes a group figure

Save this synthetic fixture as raw/input.csv:

record_id,group,value
S-101,Control,9
S-102,control,10
S-103, Control ,11
S-104,Treated,13
S-105,treated,15
S-106,TREATED,17

Save this as parameters.json:

{
  "input": "raw/input.csv",
  "output_csv": "output/summary.csv",
  "output_svg": "output/figure.svg",
  "id_column": "record_id",
  "group_column": "group",
  "value_column": "value",
  "group_order": ["control", "treated"],
  "group_labels": {"control": "Control", "treated": "Treated"},
  "expected_row_count": 6,
  "expected_summary": [
    {"group": "control", "count": 3, "total": "30.00", "mean": "10.00"},
    {"group": "treated", "count": 3, "total": "45.00", "mean": "15.00"}
  ]
}

The acceptance values come from a manual calculation made before execution: Control is 9 + 10 + 11 = 30, with mean 10; Treated is 13 + 15 + 17 = 45, with mean 15. They verify arithmetic and wiring only. They do not establish that a mean is scientifically appropriate, that the groups are comparable, or that the observed difference is meaningful.

Run the tests and analysis from the project root. On Windows PowerShell:

py -3 -m unittest steps/test_pipeline.py -v
py -3 steps/analyze.py --parameters parameters.json

On macOS or Linux:

python3 -m unittest steps/test_pipeline.py -v
python3 steps/analyze.py --parameters parameters.json

All eight tests must report ok, and the analysis must print PASS: 6 rows, 2 groups, outputs regenerated. output/summary.csv must contain the declared rows. The accessible output/figure.svg must expose an image role, title, and description, and show Treated longer than Control.

Company framing: a monthly export becomes a report figure

Use the same project and code, but begin in a separate clean copy. Replace raw/input.csv with this synthetic fixture:

record_id,group,value
R-201,North,120
R-202,north,100
R-203, North ,80
R-204,South,90
R-205,south,110
R-206,SOUTH,130

Use this parameters.json:

{
  "input": "raw/input.csv",
  "output_csv": "output/summary.csv",
  "output_svg": "output/figure.svg",
  "id_column": "record_id",
  "group_column": "group",
  "value_column": "value",
  "group_order": ["north", "south"],
  "group_labels": {"north": "North", "south": "South"},
  "expected_row_count": 6,
  "expected_summary": [
    {"group": "north", "count": 3, "total": "300.00", "mean": "100.00"},
    {"group": "south", "count": 3, "total": "330.00", "mean": "110.00"}
  ]
}

The checked result is North 120 + 100 + 80 = 300, mean 100; South 90 + 110 + 130 = 330, mean 110. The figure does not explain the difference or decide whether anyone met a target. Require the same eight test behaviours.

Write the handoff before asking for it

Create README.md for the selected framing. Use this complete minimum and replace bracketed values before committing:

# Reproducible analysis

Purpose: regenerate the [Lab group figure / Company monthly report figure] from
the included synthetic raw export.

## Requirements

- Python 3 with the standard library
- Git for identifying the tested commit
- No account, network access, credentials, or package installation

## Files

- `raw/input.csv`: immutable synthetic input; do not edit during a run
- `parameters.json`: all paths, columns, labels, order, counts, and acceptance values
- `steps/analyze.py`: validation, summary, and output generation
- `steps/test_pipeline.py`: clean-room, rerun, and failure tests
- `.gitignore`: excludes Python caches and local environments
- `output/summary.csv` and `output/figure.svg`: generated; never hand-edit

## Run from a fresh checkout

1. Record `git rev-parse HEAD`.
2. Hash `raw/input.csv` and record the before value.
3. Run the eight tests using `py -3 -m unittest steps/test_pipeline.py -v` on
   Windows or `python3 -m unittest steps/test_pipeline.py -v` on macOS/Linux.
4. Remove `output` with `Remove-Item -Recurse -Force output -ErrorAction SilentlyContinue` on PowerShell or
   `rm -rf output` on macOS/Linux. An absent folder is acceptable.
5. Run `py -3 steps/analyze.py --parameters parameters.json` on Windows or
   `python3 steps/analyze.py --parameters parameters.json` on macOS/Linux.
6. Confirm the terminal starts with `PASS`, then inspect both generated files.
7. Hash the raw file and both outputs. Record raw-after and output run-1 values.
8. Run the analysis and hashes again. Raw before/after and each output's run-1/run-2
   hashes must match respectively.
9. Run `git status --short`; only generated output may be untracked or changed.

## Expected result

[Paste the two expected summary rows from the selected framing.]
The test suite must report eight passing tests. Any failed test, different value,
changed raw hash, oral instruction, or edited output is a failed handoff.

## Limits

This pipeline proves reproducibility only for the included synthetic fixture and
recorded contract. A data owner must approve schema, grouping, units, statistic,
and use before adaptation to real data.

## Colleague handoff record

Commit: [full commit hash]
Runner and date: [name or role, YYYY-MM-DD]
Environment: [operating system and `python --version` output]
Test result: [8/8 pass or fail]
Raw SHA-256 before/after: [hash] / [hash]
Summary SHA-256 run 1/run 2: [hash] / [hash]
Figure SHA-256 run 1/run 2: [hash] / [hash]
Coaching needed: [none, or exact missing instruction]
Decision: [PASS or FAIL]

Use an approved file-hashing tool available on the colleague's operating system. Capture the raw-only command before execution, then use the three-file command after each run. On PowerShell:

Get-FileHash -Algorithm SHA256 raw/input.csv
Get-FileHash -Algorithm SHA256 raw/input.csv, output/summary.csv, output/figure.svg

On macOS:

shasum -a 256 raw/input.csv
shasum -a 256 raw/input.csv output/summary.csv output/figure.svg

On Linux:

sha256sum raw/input.csv
sha256sum raw/input.csv output/summary.csv output/figure.svg

Before handoff, delete output/, run the eight tests, run the analysis twice, and review the project. Then initialise and commit it:

git init
git add .gitignore README.md parameters.json raw/input.csv steps/analyze.py steps/test_pipeline.py
git diff --cached --check
git commit -m "Build reproducible analysis pipeline"
git status --short

Confirm no cache, environment, sensitive file, or generated output is staged. Record the full commit with git rev-parse HEAD. Give the colleague a fresh clone or clean exported project at that commit. Do not send only the script: without parameters, raw fixture, tests, and README, it is not the tested pipeline.

The colleague follows the README without a screen-share. A pass has identical raw hashes before and after, identical output hashes across runs, eight passing tests, declared summary values, and Coaching needed: none. If it fails, repair a versioned step, parameter, or instruction and repeat from a clean copy.

7. What goes wrong

Raw data is edited in place

Symptom: the pipeline now produces the expected result, but the received value that caused the mismatch has disappeared and no transformation explains it.

Fix: restore the recorded raw input. Put a justified correction in a visible step or obtain a newly versioned source file from its owner. Rerun all tests and record which input version was used.

Generated output is hand-fixed

Symptom: the figure is correct in the report, but deleting output/ and rerunning brings the old label or value back.

Fix: treat output as disposable. Repair the responsible parameter or step, delete all generated output, regenerate it, and repeat the clean-room test. Never use drawing or spreadsheet edits as an invisible final pipeline stage.

A path works on one computer

Symptom: the author can run the analysis, while the colleague receives file not found for a username, drive letter, home folder, or mounted share.

Fix: keep exercise files under one project root and resolve declared relative paths from a stable file such as parameters.json, not from the author's current directory. Test from another location.

Parameters are scattered through code

Symptom: changing a label requires editing the script, a notebook cell, and the figure; one old value survives.

Fix: move run choices to one reviewed parameter file. Search the steps for duplicate path, group, threshold, and label literals. Keep implementation constants in code, but keep analysis choices in the declared contract.

Dependencies exist only in the author's environment

Symptom: an import succeeds for the builder and fails for the colleague, or two environments format a result differently.

Fix: declare the runtime and packages. This exercise avoids external packages; a larger project should use the team's approved lockfile and environment setup. Test from a clean environment rather than installing missing pieces ad hoc during handoff.

An old output makes a broken run look successful

Symptom: the command fails, yet a plausible summary.csv remains from yesterday and is mistaken for today's result.

Fix: start the handoff without output/, stop on a non-zero command, and verify that deliberate bad input creates no output in a clean room. A file's presence is not proof that the current run made it.

The README requires oral footnotes

Symptom: the colleague succeeds only after being told where to run the command, which file to rename, or which unexplained warning to ignore.

Fix: mark the handoff failed and add the missing fact to the README. Restart from the new commit. Do not convert coaching into undocumented pipeline state.

8. Do it yourself: a 90-minute independent handoff

Minutes 0-10: choose the Lab or Company framing. Create the project tree and synthetic fixture. Write the expected row count, group order, totals, and means by hand before running code. Confirm no real or sensitive data is present.

Minutes 10-25: add parameters.json and steps/analyze.py. Trace the read, validation, normalization, summary, acceptance, and write stages. Confirm all paths are project-relative and all analysis choices live in the parameter file.

Minutes 25-40: add steps/test_pipeline.py. Run the eight tests. Inspect the clean-copy and failed-swap logic so you understand why old or partial output cannot make them pass.

Minutes 40-50: run the analysis, inspect every CSV value, and open the SVG. Hash the raw file and both outputs. Run again and prove the two output hashes are unchanged. Deliberately create a duplicate only in a temporary copy and confirm failure produces no output.

Minutes 50-60: write the README with exact commands, expected result, dependency statement, file roles, limits, and blank colleague handoff record. Delete output/ and follow the README yourself without relying on terminal history.

Minutes 60-68: review git status and the complete diff. Confirm no sensitive file, absolute path, generated cache, unrelated file, or manually corrected output is staged. Commit the selected synthetic project and record its full commit hash.

Minutes 68-83: ask a colleague to obtain a fresh copy at that commit and follow the README without coaching. They run the tests, regenerate from absent output, inspect values, hash before and after, rerun, and complete every evidence field.

Minutes 83-90: review the record together. If it failed, preserve the failed evidence, make the smallest documented repair, and schedule a complete clean rerun; do not fill evidence fields from your own machine. If it passed, keep the completed handoff record with the tested commit.

9. Exit check

Deliver exactly one artifact: one passing colleague handoff test attached to the versioned pipeline commit. Its evidence record must identify the commit and environment; report 8/8 tests; show an unchanged raw SHA-256; show byte-identical CSV and SVG hashes across two runs; reproduce the declared summary from absent output; state Coaching needed: none; and end with Decision: PASS.

The test fails if the colleague starts from the builder's working directory, uses an old output, edits raw data or output, receives an unrecorded instruction, installs an undeclared dependency, cannot identify the commit, or obtains a different result. The pipeline files support this single passing-test artifact; screenshots, the figure alone, and the builder's own successful run do not replace it.

10. Rule to remember

If a colleague cannot rerun it, it is not a pipeline.

11. Further reading & tools

Curriculum connections

Follow the authored paths

These explicit curriculum relationships complement the sequential Previous and Next links below.

Prerequisite books

Authored neighbours