Captions that are the voice
A caption should be what the narrator says, when the narrator says it. Since 13 November that is what every new short of the original programme shows: the narration's own words, two short lines at a time, each word lit as it is spoken — timed not by a guess from the script but by a speech recogniser that hears the finished voice.
exactly what the voice was sent: numbers and units as they are said
Chirp 3 HD reads each line; one audio file per line
a speech recogniser on the GPU gives every word its heard time
two lines of ≤ 24 characters, broken at pauses; the word lit
the caption equals the narration, complete and in order
what was drawn matches the word log, frame by frame
the final soundtrack heard again: each caption's distance measured
How it works#
- The words. Each line's caption is exactly what was sent to the voice — the spoken form, with numbers and units as the narrator says them — so the screen and the speaker never disagree.
- The times. When a line has been voiced, a speech recogniser (faster-whisper large-v3, on the GPU) hears it and returns a time for every word; the script's words take those heard times.
- The drawing. From the first frame, the captions show two lines of at most 24 characters, breaking at the speaker's pauses, the current word lit. The 90-second edits and short versions are cut between words, so a caption is never sliced.
- The checks. The caption text must equal the narration, complete and in order; what was drawn must match the word log frame by frame; and the finished soundtrack is heard once more to measure how far each caption is from its word.
How close it is#
| Episode | Words | Matched | Median | 95th percentile |
|---|---|---|---|---|
| Project 1, What if — the first captioned episode of Projects 1 and 2 (13 Nov) | 265 | 98.1 % | 0.01 s | 0.21 s |
| Project 2, Oddly Simulated — its first (14 Nov) | 279 | 96.1 % | 0.01 s | 0.11 s |
| Project 10, Fledge — a Studio sample (13 Nov) | — | 98.8 % | 0.01 s | 0.23 s |
| A Blender episode — the bar this format set | — | 99.3 % | 0.01 s | 0.09 s |
Where it stands#
The Studio's projects went first, after test videos of each kind, on 12–13 November; Projects 1 and 2 followed on the evening of 13 November, and every short they have made since carries word-timed captions, its 90-second edit included. Short versions, translations and the long-form films are next — the films' captions are built and wait for their test film — and the sync gate, measured on every episode today, becomes a hard gate when its thresholds are final. ChannelForge's version is built and merged, not yet deployed, behind feature flags, to be switched on for customers after a trial on the owner's workspace.