Autonomous Video Studios
HomePART II · The original programme
CHAPTER 6 · NEW SINCE THE FIRST EDITION

Captions that are the voice

A caption should be what the narrator says, when the narrator says it. Since 13 November that is what every new short of the original programme shows: the narration's own words, two short lines at a time, each word lit as it is spoken — timed not by a guess from the script but by a speech recogniser that hears the finished voice.

The words are the script's, the times are the voice's
Spoken text

exactly what the voice was sent: numbers and units as they are said

Voice

Chirp 3 HD reads each line; one audio file per line

Heard and timed

a speech recogniser on the GPU gives every word its heard time

Drawn

two lines of ≤ 24 characters, broken at pauses; the word lit

then three checks before the video may be posted
Same words

the caption equals the narration, complete and in order

Same frames

what was drawn matches the word log, frame by frame

Same moment

the final soundtrack heard again: each caption's distance measured

Figure 10. How a caption gets its timing. The words are the script's; the times are the voice's, heard on the GPU box after the narration is made, and checked again on the final soundtrack.

How it works#

  1. The words. Each line's caption is exactly what was sent to the voice — the spoken form, with numbers and units as the narrator says them — so the screen and the speaker never disagree.
  2. The times. When a line has been voiced, a speech recogniser (faster-whisper large-v3, on the GPU) hears it and returns a time for every word; the script's words take those heard times.
  3. The drawing. From the first frame, the captions show two lines of at most 24 characters, breaking at the speaker's pauses, the current word lit. The 90-second edits and short versions are cut between words, so a caption is never sliced.
  4. The checks. The caption text must equal the narration, complete and in order; what was drawn must match the word log frame by frame; and the finished soundtrack is heard once more to measure how far each caption is from its word.

How close it is#

EpisodeWordsMatchedMedian95th percentile
Project 1, What if — the first captioned episode of Projects 1 and 2 (13 Nov)26598.1 %0.01 s0.21 s
Project 2, Oddly Simulated — its first (14 Nov)27996.1 %0.01 s0.11 s
Project 10, Fledge — a Studio sample (13 Nov)—98.8 %0.01 s0.23 s
A Blender episode — the bar this format set—99.3 %0.01 s0.09 s
Distances between a caption and its spoken word, measured on the final soundtrack. Across Project 1's first captioned batch — eight episodes in all four of its kits — 95 to 99.6 % of words matched, with the 95th percentile between 0.09 and 0.21 seconds.

Where it stands#

The Studio's projects went first, after test videos of each kind, on 12–13 November; Projects 1 and 2 followed on the evening of 13 November, and every short they have made since carries word-timed captions, its 90-second edit included. Short versions, translations and the long-form films are next — the films' captions are built and wait for their test film — and the sync gate, measured on every episode today, becomes a hard gate when its thresholds are final. ChannelForge's version is built and merged, not yet deployed, behind feature flags, to be switched on for customers after a trial on the owner's workspace.