Build a Caption Pipeline Your Team Can Rerun Every Week

T09-L03 · Create media · Level 3 Builder · 17-minute read, plus 3 hours of doing

Forty talks, one person, one button

The institute records its Thursday seminar and puts the talks on the internal video page for people who missed them. Since spring the videos have captions, because a colleague with a hearing impairment asked, and because half the audience watches them on the train without sound. Sofia, the scientific coordinator, makes them. She uploads each recording to a transcription website, downloads the captions, fixes what she can, and uploads them next to the video. Then she's ill for two weeks. Nobody else knows the website, the login or which fixes she makes. When she's back, two talks have captions nobody checked, and one of them turns a speaker's "Lgr5-positive cells" into "large five positive cells" for forty minutes.

In a company it's the thirty how-to videos a software firm publishes for customers, in German. Every product release renames a few buttons, so the videos get re-recorded, and so do the captions. The support team makes them by hand, one video at a time, and after the spring release half the captions still use the old button names.

The transcription was never the hard part. The hard part is doing it the same way every week, checking it, and letting someone else take over. You are at Level 3 Builder: colleagues and customers depend on these captions, and a wrong one reaches all of them. This book shows you how to build a small caption pipeline with a test set, named reviewers and a README, so the captions keep coming when you're away.

After this you can

• Set up a manifest and a glossary so every recording has an ID, a language, a reviewer and the terms it must contain.
• Draft captions with a pinned speech-to-text model through a small script that never overwrites a draft or an approved file.
• Keep a test set of recordings with known transcripts, and a check that measures the error rate before any change goes live.
• Route every draft to a named reviewer, and publish only what a person approved.
• Hand the pipeline to a deputy with a README, and say where it stops and IT or self-hosting takes over.

Before you start

• Books: T09-L01 for AI labels and a first narrated clip. T10-L03 for pinning a Python environment with uv, which this pipeline reuses. T08-L02 for writing a script with OpenCode Desktop. The four-quadrant method is in T01-L01.
• Recordings you may process. Speakers must have agreed to recording and captioning. Before any recording goes to an outside service, check with your data protection contact which service your organisation has approved (T12-L01).
• An OpenAI API key, the password a script uses to reach OpenAI's service, in a project your organisation controls, with a monthly spend limit. If the recordings must not leave your organisation, see Choose tools for the alternative.
• Subtitle Edit or another caption editor, for reviewers.
• Two colleagues: one who records three short test clips, and one who agrees to be the deputy.
• About 3¼ hours: 17 minutes to read, about 3 hours to build and hand over, in two sessions.

Last verified 23 September 2026. OpenAI's speech-to-text service takes files up to 25 MB and costs well under a cent a minute. Data sent to it isn't used for training unless you opt in. Subtitle Edit is free and works offline. Details for experts at the end of this book.

The idea in one page

A caption pipeline has five parts. Leave one out and it depends on one person again.

Four facts shape how you build it.

1. Speech-to-text fails in predictable places. Names, gene and product names, abbreviations, numbers and "not". A caption can be 97% right and still wrong in the one word the talk is about. The glossary helps the model; the review catches the rest.
2. Models change, and so does your setup. A provider can retire a model or change it under the same name; you change the glossary or the prompt. Any of these can make captions better or worse. The test set tells you which, in five minutes, before a week of talks goes out.
3. Error rate is a number you can compare. The word error rate counts the words a person would have to change, add or delete, divided by the words in the correct transcript. It's crude, but it moves when something breaks, and it's the same number for everyone who runs the check.
4. The model drafts; a person approves. No caption goes on the video page because the script finished. It goes there because a named reviewer read it against the video.

What stays yours: the glossary, the known transcripts, the error rate you accept and the decision to publish. The model writes drafts.

Worked example: two caption pipelines, four quadrants

Both situations use the four-quadrant method from T01-L01. The lab one is told in full. Names and numbers are illustrations.

Lab: captions for the Thursday seminar

Structure it. Sofia writes down what the institute expects. "Every Thursday one talk of about 50 minutes, in English, sometimes German. Captions on the internal video page within three working days. A good result: the captions are checked by someone who knows the field, and anyone in the coordination office can run the pipeline when I'm away."

Understand it. Before building anything, she runs one past talk through the obvious route: the whole recording to the API, captions back. Three things come out. The file is too big: an hour of the recorded audio is about 55 MB, over the 25 MB limit. The jargon fails as expected: "Lgr5" becomes "large five", a speaker's name is spelled three ways, and "scRNA-seq" never appears correctly. And reviewing the captions against the video takes her most of an afternoon. She can't do that every week, and she's the wrong person for it: she doesn't know which gene names are right.

The last point changes her problem more than the other two. Back to Structure it: "The pipeline drafts; a reviewer from the speaker's host group checks, because they know the terms. Their name goes into the manifest when the talk is booked. The host group also sends me ten to twenty terms from the talk a week ahead." The size problem has a simple fix: the AV team exports the talk as mono audio at 32 kbit/s, about 14 MB an hour. They also cut it at the end of the talk, because the audience in the Q&A never agreed to be captioned.

Choose tools. She picks the OpenAI API with whisper-1, because it returns captions with timings directly, as a WebVTT file (a standard caption format, .vtt), takes a prompt with terms, handles English and German, and costs well under a euro for a talk. gpt-transcribe gives cleaner text, but no caption timings, so it's the wrong model for this job. The institute's data protection contact approves the API for talks whose speakers agreed to captioning; unpublished data shown in a talk stays out, which the booking form now asks. The alternative, if recordings must never leave the institute, is the open-source Whisper model on your own hardware; the route for that is Heidelberg AI Desk, and running it well is T09-L04. She writes the script with OpenCode Desktop, pins the environment with uv, the tool from T10-L03 that fixes which Python packages and versions the script uses, and gives reviewers Subtitle Edit, which opens a caption file next to the video.

Build it. She builds the test set first. Her colleague Malik records three clips of about two minutes each on the seminar-room microphone: one script packed with the institute's terms ("Lgr5-positive stem cells, scRNA-seq, organoids from the Kessler lab"), one with numbers and negations ("we did not see a difference at 37 degrees"), and one read with a strong accent by a colleague who volunteered. Because they read from scripts, the correct transcript is exact: it's the script. These three recordings and their scripts are the known answers.

Then the folder, the glossary and the settings:

seminar-captions/
  README.md  CHANGES.md  settings.toml  glossary.txt  manifest.csv
  transcribe.py  check.py  pyproject.toml  uv.lock
  tests/     three test clips (.mp3) and their scripts (.txt)
  incoming/  this week's audio, exported by the AV team
  drafts/    caption drafts and review-list.csv, written by the script
  approved/  written only by reviewers

# settings.toml. Change here, then run check.py before the next real talk.
model = "whisper-1"          # the only OpenAI model that returns captions with timings
glossary = "glossary.txt"
glossary_version = "2026-09"
prompt = "Scientific seminar at a biomedical research institute. Terms:"
test_language = "en"
max_wer = 0.08               # set from the first check run; see README

The glossary is a plain list, one term per line, kept short because whisper-1 accepts a prompt of at most 224 tokens (a token is a word or part of a word). Each talk's own terms go into the manifest instead, as expected_terms. The script adds them to that talk's prompt and checks the draft for them:

item_id,audio_file,language,expected_terms,reviewer
SEM-2026-09-25,sem-2026-09-25-talk.mp3,en,Lgr5|scRNA-seq|Kessler,Anna Weber

She asks OpenCode in Plan mode, where it proposes before it writes anything, for a script that reads these files, skips anything already drafted or approved, stops on files over 25 MB, and lists the expected terms missing from each draft. After review, transcribe.py:

"""Draft captions for every new recording in manifest.csv.
Run:  uv run transcribe.py      Settings: settings.toml      Needs OPENAI_API_KEY."""
import csv, hashlib, pathlib, sys, tomllib
from openai import OpenAI

HERE = pathlib.Path(__file__).parent
S = tomllib.loads((HERE / "settings.toml").read_text(encoding="utf-8"))
lines = (HERE / S["glossary"]).read_text(encoding="utf-8").splitlines()
terms = [t.strip() for t in lines if t.strip() and not t.startswith("#")]
client = OpenAI()

with open(HERE / "manifest.csv", newline="", encoding="utf-8") as fh:
    items = list(csv.DictReader(fh))
for item in items:
    audio = HERE / "incoming" / item["audio_file"]
    draft = HERE / "drafts" / f"{item['item_id']}.vtt"
    if draft.exists() or (HERE / "approved" / draft.name).exists():
        continue  # never overwrite a draft or an approved file
    if audio.stat().st_size > 25_000_000:
        sys.exit(f"STOP: {audio.name} is over 25 MB. Ask the AV team for mono 32 kbit/s.")
    fingerprint = hashlib.blake2b(audio.read_bytes(), digest_size=16).hexdigest()
    talk_terms = [t for t in item["expected_terms"].split("|") if t]
    with audio.open("rb") as f:
        vtt = client.audio.transcriptions.create(
            model=S["model"], file=f, response_format="vtt", language=item["language"],
            prompt=S["prompt"] + " " + ", ".join(terms + talk_terms))
    draft.parent.mkdir(exist_ok=True)
    draft.write_text(vtt, encoding="utf-8")
    missing = [t for t in talk_terms if t not in vtt]
    with open(HERE / "drafts" / "review-list.csv", "a", newline="", encoding="utf-8") as fh:
        csv.writer(fh).writerow([item["item_id"], item["reviewer"], fingerprint,
                                 S["model"], S["glossary_version"], "; ".join(missing)])
    print(f"{item['item_id']}: draft for {item['reviewer']}; missing terms: {missing or 'none'}")

The check runs the three test clips and compares the model's text with the scripts:

"""Transcribe the test clips and compare with their scripts.  Run:  uv run check.py"""
import pathlib, re, sys, tomllib
from openai import OpenAI

HERE = pathlib.Path(__file__).parent
S = tomllib.loads((HERE / "settings.toml").read_text(encoding="utf-8"))
lines = (HERE / S["glossary"]).read_text(encoding="utf-8").splitlines()
terms = [t.strip() for t in lines if t.strip() and not t.startswith("#")]
client = OpenAI()


def words(text):
    return re.findall(r"[\w'-]+", text.lower())


def wer(ref, hyp):
    """Word error rate: words to change, add or delete, divided by words in ref."""
    d = list(range(len(hyp) + 1))
    for i, r in enumerate(ref, 1):
        prev, d[0] = d[0], i
        for j, h in enumerate(hyp, 1):
            prev, d[j] = d[j], min(d[j] + 1, d[j - 1] + 1, prev + (r != h))
    return d[-1] / len(ref)


ok = True
for script in sorted((HERE / "tests").glob("*.txt")):
    ref = script.read_text(encoding="utf-8")
    with script.with_suffix(".mp3").open("rb") as f:
        got = client.audio.transcriptions.create(
            model=S["model"], file=f, response_format="text",
            language=S["test_language"], prompt=S["prompt"] + " " + ", ".join(terms))
    rate = wer(words(ref), words(got))
    lost = [t for t in terms if t.lower() in ref.lower() and t.lower() not in got.lower()]
    passed = rate <= S["max_wer"] and not lost
    ok = ok and passed
    print(f"{script.stem}: error rate {rate:.1%}, missing terms {lost or 'none'} "
          f"-> {'OK' if passed else 'TOO HIGH'}")
print("PASS" if ok else "FAIL: don't run real talks until this passes (README).")
sys.exit(0 if ok else 1)

Her first check fails on the term clip: the error rate is fine, but "scRNA-seq" is missing. The model writes "single-cell RNA seek". She adds the spoken form next to the term in the glossary, "scRNA-seq (single-cell RNA sequencing)", and removes a dozen terms nobody has used this year, to keep the prompt within its limit. The second run passes, with error rates between 3% and 6%. She sets max_wer = 0.08, just above the worst clip, and writes the reason into the README. From now on, any change to the model, the prompt, the glossary or the packages means running the check first, and a rate above 8% or a lost term means no real talks until someone has looked.

Then the review step. The script writes a line into review-list.csv for each draft: ID, reviewer, a fingerprint (a short code that identifies the exact audio file), model, glossary version, and the expected terms it couldn't find. Sofia sends the reviewer the draft and the video link. The reviewer opens both in Subtitle Edit, reads the captions against the talk, fixes them, saves the file into approved/ and adds their initials to the review list. Only files in that folder go on the video page, with the line "Captions drafted with AI and checked by [reviewer]."

The first real week shows the manifest paying off. The script reports missing terms: Kessler; the speaker said "the Keller lab" once and "Kessler" never, and the reviewer, from the Kessler lab, confirms the draft was right and the manifest was wrong. A missing term is a reason to listen, not a reason to change the caption.

Hand it over. The README covers setup (uv, the API key from the institute's project, never in a file), the weekly routine (manifest row, audio in incoming/, uv run transcribe.py, send drafts), the check (when and how, and what the 8% means), and When it stops, one line per STOP message. Maintainer: Sofia. Deputy: Malik, from the coordination office. The API project has a monthly spend limit of 20 euros, and the README says who can raise it.

Then the rerun. Sofia takes a week off on purpose. Malik follows the README on his own laptop: installs uv, runs uv run check.py, gets PASS, and processes that week's talk. One thing stops him: the AV team sent the full recording, 52 MB, and the script refuses it. He finds the STOP line in the README and asks for the talk-only export. The talk is captioned on Friday, reviewed by Monday, and Sofia reads about it in review-list.csv when she's back.

What it taught her: the script took an evening. Deciding who reviews, and writing the test set that tells her when something changed, took the rest, and they're why the pipeline survived her holiday.

Company: German captions for how-to videos

Structure it. Jana in customer support writes: "Thirty how-to videos in German, two to five minutes each, re-recorded after every product release. Captions within a week of the release, in the product's current words."

Understand it. She compares last release's captions with the current product. The model does fine with ordinary German. It fails on the product's own words: "Arbeitsbereich", "Freigabe-Workflow" and the product name, which it spells like an English word. Worse, the spring release renamed "Projekt" to "Arbeitsbereich", and several captions were corrected by hand to the old term, because that's what the reviewer remembered. Back to Structure it: the glossary is versioned per release and comes from the release notes, not from memory. Each video's ID includes the release, HOWTO-12-R26.3, so an approved caption can never be matched to the wrong recording.

Choose tools. The same script and model, with language = "de". The company is an OpenAI API customer and already has a project with European data residency, meaning its data is kept in Europe, which the data protection contact prefers; the README records the European address the script uses. Her test set is three videos from the last release whose captions were approved, and she re-records the narration of one with the new terms, so its script is the known answer for the new glossary.

Build it. Her first check fails: "Freigabe-Workflow" comes back as "Freigabe Workflow" in two places. She decides that's fine, because a reader doesn't stumble over a missing hyphen, and changes her term check to ignore hyphens rather than raising max_wer. That decision goes into CHANGES.md with the reason. The reviewer for each video is the product specialist for that area, named in the manifest. Her deputy runs the next release from the README.

Where it stops. After the second release, the web team asks whether approved captions could upload themselves to the help-centre video platform. That's a connection into a system the company runs, with a service account and permissions: Level 4 work with IT. Jana writes it into the README as the next step.

In both cases the model was the easy part. The test set, the glossary per version and a named reviewer are what made the captions dependable.

Where your build stops

A script, a test set, a manifest and a README are right for a team that captions a few hours a week, by hand, with named reviewers. Stop and bring in IT, or move to Level 4, when:

• recordings may not leave your organisation, such as patient consultations, participant interviews or confidential meetings. Then the model has to run on your own hardware (T09-L04, through Heidelberg AI Desk), and an ethics or data protection review comes first;
• captions should publish themselves to a video platform, learning system or website;
• the volume grows to many hours a week, where the manual review needs its own planning and budget;
• you add dubbing or AI-generated voices, which bring consent and labelling questions of their own (T09-L05).

What goes wrong

Captions published without review. Symptom: a gene name is wrong for forty minutes on the video page. Fix: only files in approved/ are published, and only reviewers write there.

The wrong reviewer. Symptom: the coordinator checks captions she can't judge. Fix: a reviewer who knows the field, named in the manifest when the recording is booked.

Changing the setup without the check. Symptom: a new glossary makes one term right and three others wrong. Fix: any change to model, prompt, glossary or packages means check.py first. A FAIL stops real work until someone has looked.

Known answers you guessed. Symptom: the check passes, but the reference transcript had errors of its own. Fix: use scripted test clips, where the script is the correct transcript, or a transcript a second person corrected against the recording.

A glossary that grows forever. Symptom: two hundred terms, and the prompt runs past the model's limit. Fix: keep the standing glossary short and put each recording's terms into the manifest.

Forcing the glossary onto the caption. Symptom: a script replaces every "Keller" with "Kessler". Fix: a missing term raises a flag for the reviewer. It never changes the text automatically.

Recordings nobody cleared. Symptom: the Q&A, with audience voices, goes to an outside service. Fix: process only what speakers agreed to, and cut the rest before the audio reaches the script.

Do it yourself (3 hours, two sessions)

1. Structure it (15 min). Write who needs captions, how fast, in which languages, and who can judge their accuracy.
2. Understand it (30 min). Run one past recording through the API by hand. List what breaks: file size, terms, languages, review time. Decide who reviews, and rewrite your problem.
3. Choose tools (15 min). Confirm with your data protection contact which service is approved for your recordings. Pick the model for captions with timings, and decide where drafts and approved files live.
4. Build it: test set and check (40 min). Record three scripted clips with your hardest terms, numbers and a negation. Set up the folder with uv, write the glossary and run check.py. Set max_wer just above your worst clip and write down why.
5. Build it: script and review (45 min). Draft transcribe.py with OpenCode, review every line, and run one real recording. Send it to the named reviewer, and publish only what comes back approved.
6. Build it: README and rerun (35 min). Write the README and set a spend limit. Let your deputy run the check and one week from the README alone. Fix the README wherever they got stuck.

Exit check

Delivered artifact: a pipeline folder and one week of captions, with:

• the settings file with the pinned model, the glossary with a version, and the manifest with a reviewer for every item;
• the drafting script, which never overwrites a draft or an approved file;
• three test clips with known transcripts, and the check script with the error-rate limit and its reason;
• the review list and the approved caption files for one week;
• a README with setup, weekly routine, check and STOP messages, a maintainer, a deputy and the spend limit.

It passes when the deputy runs the check and one week's recordings on their own computer from the README alone, the check passes, and every published caption file has a named reviewer.

Rule to remember

The model drafts, the test set tells you when it changed, and a named person approves.

Details for experts

Checked on 23 September 2026. You don't need this to follow the book; it's for setting things up or answering a colleague.

Further reading and tools

Next books

• T09-L01 · Make One Image or Narrated Clip, and Label It: labelling what AI made.
• T09-L04 · Self-hosted media models: running Whisper on your own hardware when recordings can't leave.
• T09-L05 · Scale, provenance and rights: larger volumes, AI-generated voices and rights.
• T10-L03 · Build an Analysis Your Colleague Can Rerun and Get the Same Numbers: pinning the environment with uv.
• T05-L02 · Put AI in the Middle of a Workflow, and Check Every Answer: the same review-queue idea in n8n.

Tools

• OpenCode: Taught. Writes the first draft of the script.
• Whisper (open source): Catalogued. The open model behind whisper-1, for running on your own hardware.
• Heidelberg AI Desk: Catalogued. The route when you need to run a model yourself.
• Deepgram: Catalogued. Another speech-to-text service with term prompting.
• n8n: Catalogued. When you'd rather build the same steps as a visual workflow.