Media pipelines

This mini-book is for Level 3 Builder. You will turn a folder of recordings into a traceable batch of draft transcripts and subtitles, with an optional dubbing branch and a human gate before any approved output. The exercise uses synthetic recordings only. A successful execution is not permission to publish.

2. Forty recordings, one fragile shortcut

You receive forty seminar recordings that need transcripts and subtitles. Someone proposes opening each file, clicking transcribe, downloading the result, and renaming it by hand. The first files look easy. By file twelve, you find two outputs with the same name, a speaker's invented project name spelled three different ways, and no reliable link between a subtitle and its recording. One transcript has already reached the shared presentation folder without review.

The Company version is equally risky. Forty fictional support-call recordings need internal transcripts and summaries. A fast batch can repeat the same recognition error forty times, retain more conversation than the task needs, or place a polished but wrong account reference in every summary. Speech-to-text creates candidate text; it does not establish who spoke, what they intended, or whether the result is accurate.

The answer is not slower clicking. It is a visible pipeline:

approved input folder
  -> manifest and preflight
  -> bounded processing
  -> draft transcript and subtitle
  -> review_required queue
  -> human correction and approval
  -> approved output folder

Every item keeps the same stable ID from input to output. Failures stop or enter a hold queue. Nothing in the processing branch can publish, send, overwrite the source, or approve its own result.

3. After this you can

• Build a repeatable input-process-output workflow for at least ten media items.
• Preserve the mapping between each source recording, transcript, caption file, and review decision.
• Use a glossary to surface likely errors in names, jargon, identifiers, numbers, and abbreviations.
• Estimate duration, calls, storage, and review effort before starting a large batch.
• Add and review subtitle or optional dubbing stages, releasing an item only after provenance, rights, consent, accessibility, and human checks pass.

4. Prerequisites

• T09-L02 - Consistent assets for your company or lab, for reusable production kits and source-rights records.
• T05-L02 - Automations with AI in the middle, for validation, bounded retries, and human queues.
• An approved n8n test workspace or an equivalent workflow runner with execution history.
• An approved speech-to-text service that can return a transcript and timestamped segments. Whisper or another approved model can provide the transcription capability; use the deployment and data route your organisation has approved.
• A credential stored in the workflow platform's credential store, never in a node field, manifest, filename, screenshot, or exported workflow.
• Ten short synthetic audio or video fixtures, each 10-20 seconds, and about 60 minutes.

Keep the workflow manual and disabled after the test. Do not upload real seminar recordings, support calls, participant or customer speech, unpublished findings, account details, credentials, or background conversations. Real recordings can contain personal or confidential information even when the planned topic sounds harmless. Production use requires a documented purpose, source permission, participant or speaker notice where applicable, approved processor and region, retention period, access control, and deletion path.

5. The idea in one page

Treat the manifest as the batch contract

A folder tells you what files exist. A manifest tells the workflow what each file means and how its outputs must be named. Create the manifest before calling a model.

Hashing does not prove truth or rights. It proves that later records refer to the same bytes. permission_basis does not need private legal text in the manifest; use a safe reference such as synthetic-fixture-v1 or an approved internal permission-record ID.

Use states that cannot be confused with publication:

received -> preflight_passed -> processing -> review_required
                                          -> processing_failed
review_required -> approved | held

Only a named person can set approved. The pipeline may write an internal approved-output folder after that decision, but it must not post to a website, learning platform, video host, customer system, manuscript, or public drive. Publication is a later action with its own owner and channel check.

Preflight before the expensive step

Reject the batch before transcription if an ID is duplicated, a file is missing, a hash differs, permission is unclear, a duration is implausible, or an output name would collide. Then run a two-item pilot. Record actual processed minutes, elapsed time, provider calls, output size, and human review minutes.

Estimate the full batch from measured values rather than a remembered price:

media minutes = sum(duration_seconds) / 60
maximum calls = item count * (calls per selected stage + allowed retries for that stage)
estimated service cost = media minutes * current approved rate
estimated review effort = media minutes * measured review factor
estimated storage = inputs + drafts + approved outputs + retained evidence

Check the provider's current unit, minimum charge, file limit, supported format, timestamp behavior, retention, and rate limit. Dubbing can add translation, text-to-speech, video processing, and review charges. Lip-sync can add GPU or service time. Keep those lines separate so a cheap transcription estimate does not conceal an expensive downstream branch.

A glossary is a detector, not an autocorrect list

Recognition often fails on proper nouns, abbreviations, product identifiers, unfamiliar accents, overlapping speech, and poor audio. A versioned glossary helps in three ways:

1. Supply it as model context only if the approved service supports that feature and the terms themselves are permitted to leave your environment.
2. Compare the draft with item-specific expected terms and raise a review flag when one is absent.
3. Give the reviewer the preferred spelling, pronunciation note, and source of authority.

Do not blindly replace similar words. If the audio says Aster ten, a global replacement to Aster-9 would create a false result. Preserve the raw draft, flag the mismatch, and let a reviewer listen to the relevant timestamp.

6. The worked example: one pipeline, two settings

The fictional Aster-9 Lab and Harborlight Help Company teams each have forty short recordings. The implementation is the same. The profile changes the glossary, access boundary, review owner, retention decision, and allowed output.

All source speech is synthetic. No real person, organisation, experiment, customer, product, account, or finding is represented.

Keep each product and its provenance distinct

A transcript is readable text; captions are synchronized access text checked against final audio and video; a translation is reviewed by a competent target-language reviewer; a dub is checked word for word against the approved translation. Lip-sync additionally needs portrayal permission, disclosure, and fresh visual review. None may approve itself.

For each item retain source ID and hash, permission reference, input name, processing date, workflow and glossary versions, service route and model where exposed, raw result, every draft and edit, output hashes, reviewer, date, target internal channel, disclosure, and decision. Provenance metadata supplements this run record; it does not prove permission, consent, or accuracy.

Step 1: make the batch packet

Create one working directory. It will become the single exit artifact.

media-batch-synthetic/
  incoming/          # ten synthetic source files
  manifest.csv       # source identity and permission
  glossary.csv       # preferred terms and review notes
  drafts/            # raw transcript, caption, and Company summary drafts
  review/            # one row per item, never a publication target
  approved/          # written only after human approval
  held/              # failed or rejected items
  run-record.md      # workflow version, estimate, execution and decision

Never overwrite incoming/. Name drafts from item_id: LAB-SYN-001.transcript.txt, LAB-SYN-001.captions.vtt, and LAB-SYN-001.provenance.json; the Company lane also writes CO-SYN-001.summary.json. An output generated from model wording, upload order, or an original display name can collide or lose its source relationship.

Use these ten invented fixture scripts for the Lab run. Record them yourself in an approved offline recorder or render them with an approved general synthetic voice. If you use TTS, record synthetic voice; no voice clone as the source and disclose that fact in the packet.

For the Company framing, keep IDs CO-SYN-001 through CO-SYN-010 and use parallel invented terms such as Harborlight, ticket HZ-14, no delivery date, seventeen credits, and review required. Do not put even fabricated email addresses or phone numbers into the audio; they add no value to this test.

Create manifest.csv with one row per actual media file:

item_id,source_file,source_sha256,duration_seconds,language,expected_terms,permission_basis,output_base
LAB-SYN-001,LAB-SYN-001.wav,REPLACE_WITH_ACTUAL_SHA256,12,en,"Aster-9",synthetic-fixture-v1,LAB-SYN-001
LAB-SYN-002,LAB-SYN-002.wav,REPLACE_WITH_ACTUAL_SHA256,14,en,"Quenby|fourteen",synthetic-fixture-v1,LAB-SYN-002

Add the remaining eight rows. Replace hashes and durations with measured values. Do not submit placeholders as run evidence.

Create glossary.csv:

term,preferred_form,pronunciation_note,authority,version
aster nine,Aster-9,"as-ter nine",synthetic fixture script,GLO-1
quenby,Quenby,"kwen-bee",synthetic fixture script,GLO-1
q z four,QZ-4,"cue zee four",synthetic fixture script,GLO-1
delta v,Delta-V,"delta vee",synthetic fixture script,GLO-1

Step 2: build the manual workflow

In an approved n8n test workspace, create a disabled workflow named Media batch - synthetic review gate. Equivalent runners are acceptable if they expose the same data and enforced branches.

1. Start with Manual Trigger and read the ten manifest rows from an approved test location.
2. Validate unique IDs, file references, hashes, positive durations, permission references, and unique output bases. Route any failure to held; do not call the media service.
3. Use Loop Over Items or your runner's bounded iterator. Set a small batch size that respects the approved provider's current limits.
4. Fetch exactly the file named by the current row. Confirm the binary item still carries item_id, hash, duration, and expected terms.
5. Send the media to the approved transcription operation using a stored credential. Request timestamped segments if supported. Do not map unrelated execution data.
6. Attach the untouched manifest fields to the provider result. Never ask the model to recover the source ID from speech.
7. Validate the returned text and timestamps. Create a WebVTT draft only from validated segments.
8. In the Company lane only, create a structured summary draft from the validated transcript as described below.
9. Write transcript, caption, optional Company summary, and provenance drafts under the stable output base, then set status: review_required for every successful item.
10. Route technical failures to held with item ID, stage, safe error category, attempt count, and time. Keep both final branches internal; there is no publish, send, upload, customer-update, or manuscript node.

If the provider is unavailable or rate-limited, retry only according to an approved finite policy. A retry must reuse the same item_id, source hash, and run ID. Do not retry a completed item merely because a later node failed; resume from recorded state or hold it. This prevents duplicate calls and mismatched outputs.

Step 3: validate and create caption drafts

Configure an n8n Code node to run once for each item. Map the provider response to transcription.text and transcription.segments, where each segment has numeric start, numeric end, and string text. Adapt only that mapping for the approved provider; preserve the validator contract.

function validateAndFormat(source) {
const text = source.transcription?.text;
const segments = source.transcription?.segments;
const duration = Number(source.duration_seconds);
const errors = [];

if (typeof source.item_id !== 'string' || !source.item_id.trim()) errors.push('missing item_id');
if (typeof text !== 'string' || !text.trim()) errors.push('empty transcript');
if (!Number.isFinite(duration) || duration <= 0) errors.push('invalid duration_seconds');
if (!Array.isArray(segments) || segments.length === 0) errors.push('missing segments');

const safeSegments = Array.isArray(segments) ? segments : [];
let previousEnd = 0;
for (const [index, segment] of safeSegments.entries()) {
  if (!segment || typeof segment !== 'object' ||
      !Number.isFinite(segment.start) || !Number.isFinite(segment.end) ||
      typeof segment.text !== 'string' || !segment.text.trim()) {
    errors.push(`invalid segment ${index}`);
    continue;
  }
  if (segment.start < 0 || segment.start < previousEnd || segment.end <= segment.start) {
    errors.push(`non-monotonic segment ${index}`);
  }
  if (Number.isFinite(duration) && segment.end > duration + 1) {
    errors.push(`segment ${index} exceeds source duration`);
  }
  previousEnd = segment.end;
}

const expected = String(source.expected_terms || '')
  .split('|').map((term) => term.trim()).filter(Boolean);
const missingTerms = expected.filter((term) =>
  !String(text || '').toLocaleLowerCase('en').includes(term.toLocaleLowerCase('en'))
);

const vttTime = (seconds) => {
  const total = Math.round(seconds * 1000);
  const hours = String(Math.floor(total / 3600000)).padStart(2, '0');
  const minutes = String(Math.floor(total / 60000) % 60).padStart(2, '0');
  const secs = String(Math.floor(total / 1000) % 60).padStart(2, '0');
  const millis = String(total % 1000).padStart(3, '0');
  return `${hours}:${minutes}:${secs}.${millis}`;
};
const captionsVtt = errors.length ? null : `WEBVTT\n\n${safeSegments.map(
  (segment, index) => `source-segment-${index}\n${vttTime(segment.start)} --> ${vttTime(segment.end)}\n${segment.text.trim()}`
).join('\n\n')}\n`;

return {
    ...source,
    validation_ok: errors.length === 0,
    validation_errors: errors,
    review_flags: missingTerms.map((term) => `check expected term: ${term}`),
    captions_vtt: captionsVtt,
    status: errors.length === 0 ? 'review_required' : 'held',
  };
}

return {json: validateAndFormat($json)};

This check detects structure and obvious timing errors. It cannot decide that recognized words are faithful to audio. A perfect structural result still enters human review. Likewise, a missing expected term is a flag, not proof that the model is wrong; the script, audio, or manifest may be the part that needs correction.

For {start: 0, end: 1.25, text: "Welcome to Aster-9."}, the exact caption output is:

WEBVTT

source-segment-0
00:00:00.000 --> 00:00:01.250
Welcome to Aster-9.

Run six node fixtures before connecting a provider: one valid item must return review_required and that exact cue; start: NaN, end: NaN, and duration_seconds: "malformed" must each return held; object-valued segments must return held with missing segments; and [null] must return held with invalid segment 0. None may throw. Keep model segments as source-linked drafts, not automatically readable captions; a reviewer may split or combine cues while retaining the source-segment relationship.

Create and validate the Company summary draft

Only the Company lane calls an approved text operation, using the validated synthetic transcript. Require exactly:

{
  "item_id": "CO-SYN-001",
  "summary": "Fictional Harborlight request mentions ticket HZ-14 and gives no delivery date.",
  "evidence_segments": [0],
  "status": "review_required"
}

Use this deterministic validator before the human evidence check:

function summaryErrors(draft, itemId, segmentCount) {
  const keys = ['evidence_segments', 'item_id', 'status', 'summary'];
  if (!draft || typeof draft !== 'object' ||
      JSON.stringify(Object.keys(draft).sort()) !== JSON.stringify(keys)) {
    return ['summary keys do not match schema'];
  }
  const errors = [];
  if (draft.item_id !== itemId) errors.push('summary item_id mismatch');
  if (typeof draft.summary !== 'string' || !draft.summary.trim()) errors.push('empty summary');
  if (draft.status !== 'review_required') errors.push('invalid summary status');
  if (!Array.isArray(draft.evidence_segments) || !draft.evidence_segments.length ||
      !draft.evidence_segments.every((index) => Number.isInteger(index) && index >= 0 && index < segmentCount)) {
    errors.push('invalid evidence_segments');
  }
  return errors;
}

Extra keys, a mismatched ID, empty text, or nonexistent segment indexes go to held without a model retry. The support quality owner then rejects any name, number, date, promise, or action absent from the cited transcript segments. Test the exact supported summary above and a draft that invents a delivery date; the latter must be held by the reviewer even when its schema passes. Store both results. The summary cannot update a case or contact a customer.

Step 4: review all ten items

Create one review row per item with these fields:

item_id | source_sha256 | transcript_file | captions_file | summary_file | glossary_version
flags | transcript_edits | caption_edits | summary_edits | accessibility_check | disclosure
target_internal_channel | reviewer | reviewed_at | decision | approved_output_sha256

The Lab reviewer listens to every synthetic clip while reading its transcript. The Company reviewer does the same, then verifies every Company summary statement against its cited transcript segments. For each item, reviewers check beginning and end, flagged timestamps, names, identifiers, quantities, negation, omissions, and sound cues. They then check caption timing, readable breaks, punctuation, speaker labels where needed, and correspondence with final audio.

Require the reviewer to choose approved or held, add their identity and time, and hash the corrected outputs. A blank reviewer, a bulk approval with no item evidence, or a model-generated approved value fails the gate. If the transcript changes, regenerate or edit captions and review the encoded media again. The caption and transcript must describe the final version, not an earlier audio draft.

For these fixtures, use disclosure such as Synthetic exercise audio transcribed with AI; transcript and captions reviewed by [role]. If a general synthetic voice created the input, say so: Synthetic exercise audio uses a general AI voice; AI transcript and captions reviewed by [role]. The label records what occurred. It does not grant rights or certify accuracy.

Step 5: add subtitles or optional dubbing

For an existing synthetic video, attach the reviewed caption file as a selectable track where the target platform supports it. Also retain the transcript and a listing description, for example: Fictional Aster-9 training clip explaining that human review is required before output; synthetic voice and AI-assisted captions. Test the final video muted, with audio, at the intended size, and under the likely crop or player controls.

If you test dubbing, use only one approved synthetic clip. Begin from the approved source-language transcript, create a draft translation, and obtain a separate language review. Then synthesize with an approved general voice. Do not clone a seminar speaker or caller. Compare the final dub word for word with the approved translation, inspect timing and mix, provide reviewed captions in the delivered language, retain the source-language transcript, and update the disclosure and provenance record.

Do not add lip-sync merely because a tool offers it. If the learning goal can be met with dubbed audio and captions, stop there. A lip-synced version creates a new manipulated visual asset and needs a new portrayal permission decision, frame review, disclosure check, output hash, and approval. Keep it in review_required until those checks are complete.

Step 6: scale the estimate to forty

Record the two-item pilot separately from the ten-item acceptance run. Suppose the pilot contains 0.5 media minutes, takes 1 minute of service processing, and takes 2 minutes of human transcript and caption review. Those invented figures are an illustration, not a performance claim. For the real exercise, use your observed values.

Scale with the actual total duration of all forty files, not 40 x average when the manifest already contains exact durations. Add expected rework, translation review, and dubbing review as separate lines. Compare the estimate with an agreed budget and deadline before submitting the batch. If the estimate exceeds either limit, reduce scope, split the batch, or stop. Do not discover the cost boundary after processing thirty-nine files.

The worked pipeline is complete when all ten test items preserve their source mapping, every successful item enters review_required, every Company item has a cited summary decision, at least one deliberate glossary mismatch is visible, one simulated technical failure reaches held, and only individually reviewed items appear under approved/.

7. What goes wrong

There is no glossary

Symptom: Aster-9, Quenby, and QZ-4 become different plausible words across the batch.

Fix: version a small authoritative glossary, associate expected terms with item IDs, and show likely misses at the relevant timestamps. Never global-replace without listening.

Output loses its source

Symptom: files are named transcript-final-7.txt, and nobody can prove which recording or processing run produced them.

Fix: assign the ID before processing, retain the source hash, derive every output base from that ID, and reject duplicate IDs or names during preflight.

The batch publishes its own drafts

Symptom: a successful transcription node sends captions to a public video or writes text into a shared manuscript.

Fix: end automation in review_required. Make human identity, decision, corrected-output hash, channel, and review time mandatory before an internal approved copy can be created. Keep publication outside this exercise.

A high score replaces listening

Symptom: outputs with provider confidence or no glossary flags are approved without opening the media.

Fix: treat scores and flags as review prioritization only. Listen to every item in this ten-item exercise and check terms, numbers, negation, omissions, timing, and meaning.

Cost and retry behavior are decided after the run

Symptom: forty hours of media enter several paid stages before anyone totals the units, and a timeout reruns completed transcription or attaches a later caption to the wrong draft.

Fix: measure a two-item pilot, price each stage, set call and retry ceilings, and obtain the budget decision first. Use stable item and run IDs, record completion by stage, reject stale results, and never infer identity from execution order.

Raw transcript becomes captions

Symptom: cues are too long, speaker changes disappear, meaningful sounds are missing, and timing follows model segments rather than readable phrases.

Fix: use timestamps only as a draft. Edit captions against the final encoded audio and check reading, synchronization, punctuation, speakers, sound cues, contrast, safe areas, and muted playback.

Dubbing hides a changed claim

Symptom: translation changes not approved to approved, while fluent synthetic speech makes the error sound authoritative.

Fix: approve the source transcript first, review the translation separately, compare the final dub with approved text, and rerun caption, disclosure, provenance, and release checks.

Access and retention copy the whole source forever

Symptom: raw calls, transcripts, temporary audio, and provider responses remain in broad execution logs with no deletion date.

Fix: minimize mapped fields and logs, restrict the packet, apply the approved retention schedule, verify provider and workflow deletion paths, and retain only evidence justified by the stated purpose.

8. Do it yourself: a ten-item batch in 60 minutes

Minutes 0-6: choose the Lab or Company framing. Create one media-batch-synthetic/ packet and name the owner, reviewer, allowed internal output, forbidden actions, approved tool route, and stop control.

Minutes 6-14: create ten 10-20 second synthetic recordings from the fixture pattern. Build the full manifest with unique IDs, actual filenames, hashes, measured durations, language, expected terms, permission reference, and output bases. Add a four-term glossary.

Minutes 14-20: preflight all rows. Deliberately duplicate one output base and confirm the batch stops before transcription. Repair it. Estimate total media minutes, maximum calls, current service cost, storage, and review minutes from a two-item pilot.

Minutes 20-34: build the manual loop, approved transcription call, mapping, validator, WebVTT formatter, optional Company summary validator, review_required queue, and held branch. Keep every destination internal.

Minutes 34-40: run all ten items. Deliberately misspell one expected term in the manifest or inject one synthetic malformed segment result; confirm the item is flagged or held without losing its ID. Preserve the execution evidence, then correct the test fixture and rerun only the appropriate stage.

Minutes 40-51: review every transcript against its audio. Correct names, identifiers, numbers, negation, and omissions. Edit captions for timing and readability; in the Company lane, check every summary claim against its cited segments. Retain raw drafts and record edits.

Minutes 51-56: approve or hold each item individually. Add reviewer, time, decision, disclosure, target internal channel, and corrected-output hash. Confirm an unreviewed item cannot appear in approved/.

Minutes 56-60: inspect the packet as another person would. Select one final video if available and check subtitles with sound on and off. Verify source-to-output mapping, held-item evidence, glossary version, accessibility files, provenance, call and cost estimate, and absence of credentials or real data. Leave publication disconnected.

9. Exit check

Deliver exactly one artifact: one media-batch-synthetic/ packet containing a completed batch run over at least ten items with an enforced human review gate before approved output.

It passes when the packet contains the ten permitted synthetic sources or safe references to them, a complete manifest and hashes, glossary, measured pilot and forty-item cost estimate, workflow version or export, execution evidence, raw transcript and caption drafts, Company summary drafts when that lane is selected, one deliberate flag or held failure, item-level review records, corrected accessible outputs, disclosures, provenance, and approved-output hashes. Every output must map to one source ID and hash. Every successful automated result must first have status review_required, and every file under approved/ must show a named human decision.

It fails if any item is published or sent, any real or unapproved recording is used, an ID mapping is missing, a credential appears, a model or bulk default approves output, captions are not checked against final audio, or an uncertain rights, consent, privacy, translation, identity, or factual issue is marked passed. Revise the same packet rather than submitting a second artifact.

10. Rule to remember

Batch the work, not the judgement.

11. Further reading & tools

• Taught: T09-L02 - Consistent assets for your company or lab - preserve a reusable production kit, source rights, and release record across many assets.
• Taught: T05-L02 - Automations with AI in the middle - validate uncertain output, bound retries, and route failures to a human queue.
• Taught: Realtime voice agents - separates candidate recognition from deterministic state, cancellation, approval, accessibility, privacy, and handoff.
• Taught: Voice: text to speech and speech to text - treats recognized text as a draft and keeps consent, confirmation, retention, and non-voice access separate.
• Taught: AI video generation - reviews final captions, disclosure, rights, provenance, and channel-ready exports.
• Taught: n8n - builds manual workflows with inspectable item data, internal branches, and protected credentials.
• Catalogued: AI avatars - optional follow-up for identity, voice, portrayal, dubbing, and lip-sync boundaries.
• Catalogued: OpenAI Whisper repository (opens in a new tab) - primary model and code reference; verify the exact deployment, supported formats, license, compute, and data route you use.
• Catalogued: n8n looping guidance (opens in a new tab) - current primary guidance for item iteration and Loop Over Items behavior.
• Catalogued: ElevenLabs dubbing documentation (opens in a new tab) - current provider guidance for optional dubbing; verify present terms, consent controls, data handling, supported languages, and review behavior before use.
• Catalogued: LatentSync repository (opens in a new tab) - primary project reference for optional lip-sync; model availability does not establish identity permission, output rights, fitness, or publication approval.
• Catalogued: W3C Making Audio and Video Media Accessible (opens in a new tab) - primary accessibility guidance for captions, transcripts, descriptions, players, and media production.
• Catalogued: C2PA specifications (opens in a new tab) - version-neutral index for signed provenance specifications and their limits; record the version actually used.
• Catalogued: T09-L04 - Self-hosted media models - continues from an approved service route to operating media models on controlled hardware.
• Catalogued: Tools index - compare current media and workflow tools only after defining the input permission, data route, batch contract, budget, and review gate.