Search & explore

• Level L05
• T03
• 21 min read

A prompt many people depend on drifts: new kinds of input, a model swap, a failure nobody saw. Grow a regression set from real failures, gate every change on a rule written in advance, watch three signals, handle incidents, and hand it all to a substitute who has run it.

Matching textT03-L05 · Prompting & context · Level 5 Operator · 19-minute read, plus 4 hours of doing Keep a Production Prompt Healthy for Months, and Hand It Over Nothing changed, and the numbers moved

• Level L04
• T03
• 22 min read

When a prompt feeds a real database or a real inbox, every change is a release. Put the prompt and its tests in a repository on your code server, let every change pass an automatic test run and a named reviewer, release by tag, and roll back in minutes.

Matching textT03-L04 · Prompting & context · Level 4 Integrator · 18-minute read, plus 4 hours of doing Release Prompt Changes Through Review, Tests and a Way Back Two copies of the same prompt The clini

• Level L03
• T03
• 18 min read

A prompt colleagues rely on needs tests before anyone changes it, or swaps the model under it. Write down what the output must be, collect 15 cases with known answers, run them in Promptfoo, change one thing at a time, and leave a folder a colleague can rerun with one command.

Matching textT03-L03 · Prompting & context · Level 3 Builder · 17-minute read, plus 3 hours of doing Put a Test Set Under the Prompt Your Team Depends On The improvement nobody tested A PhD student

• youtube

Understand Gitea Actions runners and automated workflows. Not required-check enforcement, restricted AI review or rollback. Retain existing rejection of yt-jcode-wh5Sfn9LQxQ.

Matching textT03-L04: Watch 7:04–11:35 for enabling Actions, running an example and creating a workflow—the mechanism behind automatic prompt tests. This sponsored 2024 tutorial does not implement the book's…

• youtube
• 17 min video

Express desired model behaviour as explicit test cases and output assertions. Offline evaluation is not a per-item runtime guard; recording-era cost/latency ratios are not general claims.

Matching textT03-L03: See file-based Promptfoo cases, assertions and uncached comparisons. The 2023 demonstration compares two OpenAI model configurations; follow the book's pinned setup, critical-case rules…

• youtube

Construct prompts with task context, examples and output formatting. Not the book's one-change five-input test.

Matching textT03-L01: Learn how task context, examples and output formatting shape a reusable prompt. This 2025 Anthropic presentation complements the book's composition skill; use the book for the one-change-at-a-time…

One-page diagram to use with T03-L03, "Put a Test Set Under the Prompt Your Team Depends On".

One-page diagram to use with T03-L01, "Write a Prompt You Can Reuse".

One-page diagram to use with T03-L03, "Put a Test Set Under the Prompt Your Team Depends On".