--- name: caption-video metadata: {packVersion: 1.0.0} description: >- Burn timed, styled captions into a finished video on the local machine, without re-rendering the source. This is the manual fallback to the API's captioning endpoint, POST /v1/captions, which stays the default path: come here when the style the user wants is outside the API's presets, when the wording has to be hand-corrected before it is burned into the pixels, or when the source was rendered with audio disabled and the API refuses it. Transcribes the audio with Whisper, groups words into reading phrases, renders the captions in a real browser via HyperFrames, and composites them over the source with ffmpeg using a real alpha channel. Use when the user has an MP4 with speech and asks to add captions, subtitles, burned-in text, TikTok-style captions or word-by-word text to it, and the API path has been ruled out. Runs out of band, so it costs no credits and works on any video file whoever generated it. Not for generating video (use the video skills) and not for a separate .srt/.vtt sidecar file. --- # Caption a finished video Takes a finished MP4 with speech and returns the same video with timed captions burned in. Nothing is regenerated: the source pixels and the source audio pass through untouched, and the captions are composited on top. **This skill makes no API calls.** It is ffmpeg + Whisper + HyperFrames on a local file, so it costs no credits and needs no API key. ## Check the API path first `POST /v1/captions` is the default, and it is one call against a finished asset with a set of preset styles and no local toolchain at all. Read the `novoads-api` skill's SKILL.md (*Burned-in captions*) for the call, and [prompting/guide.md](prompting/guide.md) here for the full side by side. **Offer both paths rather than picking silently**, because the API costs credits and this skill costs a first-time setup. > **REST key required. A Novoads MCP connector is not a substitute.** If > `NOVOADS_API_KEY` is missing or still the placeholder, stop before any > generation work and tell the user: "Before continuing, create an API key at > and paste it into `.env`." > That holds even when `mcp__novoads__*` tools are connected and authenticated in > the session. Never call `mcp__novoads__*` tools from this repo's workflows: they > are a different surface with different behavior, including the units they quote > costs in. Repo installs verify with `./scripts/check-novoads-env.sh`; a solo > install checks `NOVOADS_API_KEY` in the environment. **Pack version.** Every `/v1` response carries `X-Novoads-Pack-Version`; mention a newer pack at only when that header names a version NEWER than this file's `metadata.packVersion` — equal or older is nothing to say, and it is never a reason to stop. Come here instead when the user: - wants a caption look the presets do not cover, - has to hand-correct the words before they are burned in. Invented brand names are the usual reason: transcription mishears them, and the API gives you nowhere to fix it, - is captioning enough footage that per-minute credits add up, - has a line the API path burns **wrong**. Its renderer can truncate a word even when its own transcript is correct — see *Safety rules* below. There is nothing to fix on that path, so the local burn is the fix, - or has a source rendered with `audioEnabled: false`, which the API refuses with a `409`. This skill can still caption it if the user supplies the words. ## When to use this skill Trigger on phrases like: - "add captions to this video" / "burn in subtitles" - "TikTok-style captions" / "word-by-word captions" - "caption this ad before I deploy it" - the captions step of a video-ad pipeline, after the master is stitched Do **not** use it to *generate* video — that's the `novoads-api` skill and the style pipelines. Do not use it when the user wants a sidecar `.srt`/`.vtt` subtitle file rather than pixels; Whisper alone covers that. ## Read order 1. **This file** — the pipeline shape, the gates, the hard-won rules. 2. **[prompting/guide.md](prompting/guide.md)** — the full recipe: Whisper model choice, the silence-offset transcription pattern, the word-grouping helper, the composition skeleton, the alpha composite, and the pitfall table. ## Prerequisites - **ffmpeg** on PATH. Homebrew's build is fine here — this skill deliberately avoids the `subtitles`/`drawtext` filters, which Homebrew ships without. - **Whisper** — `pip install openai-whisper`, or the bundled `npx hyperframes transcribe`. - **Node + npx** for HyperFrames. - **The finished video on disk.** Chat-pasted files are not accessible; ask for a real path. ## Workflow 1. **Trim dead space first, if any.** If the source has beats where VO ends and silent visual continues, tighten it *before* transcribing. Timestamps taken from an untightened master drift once the video is trimmed later. 2. **Initialize** a captioning project beside the source video. 3. **Transcribe** — pick the Whisper model by audio type (`medium.en` for the typical speech-over-music ad). **Transcribe a silence-trimmed copy and offset the timestamps back**; Whisper smears the first words across leading silence otherwise. The source file itself is never modified. 4. **Group** words into 3–5 word reading phrases, then eyeball `groups.json`. 5. **Render** the captions over a transparent background at the source's frame rate, and **composite** with ffmpeg using the alpha channel. 6. **Verify** the output: duration, resolution and frame rate match the source, audio is bit-identical, and the captions land on the words. The guide carries the exact commands for each step. ## Rules that cost a whole render to rediscover - **Composite with a real alpha channel, not a chroma key.** Render `--format mov` and let ffmpeg `overlay` use the alpha. A magenta key cannot cleanly remove a soft `text-shadow` — the shadow blends toward the key colour and leaves a purple halo on every glyph. Verified on a live clip 2026-08-03. - **Never put `