oma-video · diff
git:20260905.2ffa537 to git:20260905.6c916c9
15 added, 15 removed. Audit A to A.
---
name: oma-video
- description: Short-form, explainer, and demo video generation via a key-optional 3-tier router. Composes scripts, oma-voice narration, oma-image/oma-slide/stock visuals, key-free captions, and a per-run agent-authored Remotion composition (always-latest Remotion + remotion-dev/skills) into reproducible run directories. Routes three modes — shorts/reels (9:16), explainer (16:9 README/code/data), and demo/walkthrough (screen capture, incl. supervised headed web-app capture of any URL). Use for video, shorts, reels, short-form, demo, explainer, walkthrough, screencast, web capture, video generation, 영상, 숏폼, 쇼츠, 릴스, 데모, 설명 영상.
+ description: Short-form, explainer, and demo video generation via a key-optional 3-tier router. Composes scripts, oma-voice narration, oma-image/oma-slide/stock visuals, key-free captions, and a per-run agent-authored Remotion composition (always-latest Remotion + remotion-dev/skills) into reproducible run directories. Routes three modes — shorts/reels (9:16), explainer (16:9 README/code/data), and demo/walkthrough (screen capture from human-recorded video). Use for video, shorts, reels, short-form, demo, explainer, walkthrough, screencast, web capture, video generation, 영상, 숏폼, 쇼츠, 릴스, 데모, 설명 영상.
---
# Video Agent - Short-form, Explainer & Demo Router
## Scheduling
### Goal
Generate finished `.mp4` videos through a key-optional, 3-tier (CLI-first / MCP / guided) provider router while preserving deterministic asset buses (script -> timing -> render-spec), reproducible manifests, cost controls, and capture-path safety.
### Intent signature
- User asks for a short-form video, shorts/reels clip, TikTok/YouTube Short, explainer, demo, walkthrough, or screencast.
- User wants a topic, README, code, or data turned into a narrated, captioned video.
- Another skill needs shared video-generation infrastructure (script -> assets -> render).
### When to use
- Generating short-form video (shorts / reels) from a topic or brief (`--mode shorts`, 9:16)
- Generating an explainer from a README, code, or data set (`--mode explainer`, 16:9 / 9:16)
- Producing a demo / walkthrough from a screen capture file (`--mode demo --source file`, 16:9)
- - Supervised headed web-app capture of any URL (`--mode demo --source web --url <url>`) — a human drives the on-screen flow; the tool only opens a headed browser and records. Example categories are equal and illustrative only: demo, walkthrough, onboarding clip, bug repro, app-review screencast.
+ - Guided screen capture for demos, walkthroughs, onboarding clips, bug reproductions, and app reviews. Supply the recording with `--capture <path>`; `--source web --url <url>` supplies URL context.
- Re-rendering an existing run deterministically from `render-spec.json`
- Other skills needing video-generation infrastructure (shared invocation via `--format json`)
### When NOT to use
- Generating a single still image -> use `oma-image`
- Generating a slide deck / presentation -> use `oma-slide` (this skill calls it internally for explainer frames)
- Generating speech audio only (no video) -> use `oma-voice`
- Non-linear video editing of an existing finished mp4 -> out of scope (OpenCut-MCP deferred)
- - Supervised headed web capture is in-scope (`--source web`); live streaming is out of scope
+ - Human-recorded web flows are in scope (`--source web`); live streaming is out of scope
- Interactive HTML explainer document (not a video) -> use `oma-explanation`
### Expected inputs
- A brief (topic / README path / data) plus optional mode, aspect, locale, captions, visual, voice, music, duration, compositor, capture path, seed
- For `demo` `--source file`: a screen-capture file path (`--capture`) or Cap availability
- - For `demo` `--source web`: a target `--url` (any URL — local/staging/prod), optional `--device`/`--ready-selector`/`--show-cursor`/`--polish`/`--capture-timeout`; capture size is derived from `--aspect`/`--device` (no hardcoded size); an available browser capture runtime + an interactive TTY (else the run falls back to the guided protocol)
+ - For `demo --source web`, supply `--url` as context and `--capture <path>` as recording input; without a recording the command returns guided capture instructions.
- Authentication/environment state for oma-voice (Voicebox MCP), oma-image vendors, and optional Pexels / Pixelle keys
### Expected outputs
- A run directory under `.agents/results/videos/<timestamp>-<shortid>-<mode>/`
- Deterministic asset bus: `script.json`, `timing.json`, `render-spec.json`
- `audio/`, `visuals/`, `captions.srt` / `captions.vtt`, the rendered `<mode>-<slug>.mp4`
- `manifest.json` with providers, asset hashes, cost breakdown, and exit code
```yaml
outputs:
- name: video
description: rendered <mode>-<slug>.mp4 in the run directory
artifact: ".agents/results/videos/*/*.mp4"
required: true
- name: manifest
description: reproducibility record (providers, asset sha256 hashes, cost breakdown, exit code)
artifact: ".agents/results/videos/*/manifest.json"
required: true
- name: render-spec
description: deterministic compute boundary consumed by `oma video render <runDir>`
artifact: ".agents/results/videos/*/render-spec.json"
required: true
- name: captions
description: key-free caption tracks aligned to timing.json
artifact: ".agents/results/videos/*/captions.srt"
required: false
```
### Dependencies
- `oma video generate` CLI + central error module (exit codes aligned with `oma search fetch`)
- oma-voice (Voicebox MCP), oma-image, oma-slide as key-free fallback providers
- Per-run Remotion project at `<runDir>/remotion/` scaffolded by `oma video compose` on the always-latest toolchain (`~/.cache/oma-video/remotion/<ver>/`); the composition is agent-authored per run following remotion-dev/skills (`resources/remotion-authoring/`)
- `resources/vendor-matrix.md`, `resources/execution-protocol.md`, `resources/prompt-tips.md`, and the `video:` section of `.agents/oma-config.yaml`
### Control-flow features
- Branches by mode (shorts / explainer / demo), aspect, visual strategy, provider availability, cost threshold, capture requirement, and path safety
- Runs a per-capability fallback chain (real key/resource -> key-free fallback) per backend rule 11
- Reads briefs/captures and writes assets, render-spec, and manifests
- Calls external resources: Voicebox MCP, oma-image vendors, Remotion toolchain, optional Pexels / Pixelle
## Structural Flow
### Entry
1. Validate that the brief carries enough mode/topic signal (or infer the mode from keywords).
2. For `demo`, confirm a capture path exists (or Cap is available); otherwise enter the guided protocol.
3. Resolve defaults from shipped code defaults -> the `video:` section of `.agents/oma-config.yaml` -> env vars -> CLI flags; check output path safety and limits.
### Scenes
1. **PREPARE**: Resolve mode/aspect/locale, clarify or amplify the brief, choose the visual + compositor strategy.
2. **ACQUIRE**: Probe provider availability (voice / visual / caption / compositor), validate capture path, check cost.
3. **ACT**: Run the mode pipeline — script -> (voice ∥ visuals ∥ captions) -> render-spec -> compositor render.
4. **VERIFY**: Validate every asset-bus schema, manifest hashes, exit code, and the output mp4.
5. **FINALIZE**: Return the run-dir path, the mp4 path, and any provider/coverage warnings.
### Transitions
- If the brief lacks a clear mode, infer from keywords (shorts/reels -> shorts; README/code -> explainer; capture -> demo) and show the user the inferred plan before generating.
- If the selected visual provider key is absent (Pexels / Pixelle), fall through the chain to the key-free oma-image stills + Ken Burns fallback and annotate coverage.
- - If `demo` `--source web` has a `--url`, dispatch the headed web-capture path (human-driven flow, ENTER to stop); if the browser capture runtime is unavailable OR there is no interactive TTY, fall back to the guided protocol (no hang).
+ - For `demo`, ingest `--capture <path>` or provide guided capture instructions. `--source web` requires `--url` as context; it does not start a browser recorder.
- If `demo` `--source file` has no capture and Cap is unavailable, emit the guided capture protocol and stop (exit code maps to capture-required).
- If estimated cost (Pixelle / RunningHub credits) exceeds the guardrail, require confirmation unless bypassed.
### Failure and recovery
- If a provider is unavailable, try the next provider in the capability's `order`; only chain exhaustion is a stage failure.
- Remotion is always the latest npm release and oma owns NO composition code: `oma video compose <runDir>` refreshes the toolchain + remotion-dev/skills (throttled) and scaffolds `<runDir>/remotion/`; you author `src/Root.tsx` per its `AUTHORING.md` and `resources/remotion-authoring/<mode>.md`, then `oma video render <runDir>`. A tsc/render failure is a composition bug: re-read the skills, fix, re-render (no fixed cap; stop only after two consecutive attempts without progress and report).
- If the toolchain cannot be fetched (offline, nothing cached), point the user to `oma video doctor --install` once online; fall back to the MPT compositor where applicable (MPT itself needs a one-time `oma video doctor --install-mpt`).
- If Voicebox MCP is down, fall back to estimated timing (still produces captions). A whisper.cpp hop between the two is reserved but not yet wired (`TODO(oma-deferred): whisper-cpp`).
- If the brief locale is non-source, translate via oma-translation (key-free); if absent, warn and keep source text.
### Exit
- Success: `<mode>-<slug>.mp4` and `manifest.json` exist in the run directory; all schemas validate.
- Partial success: video renders with a key-free fallback in place of a paid provider; coverage is annotated in warnings.
- Failure: no video is produced and the route/cost/capture/auth/safety blocker is explicit in the exit code + manifest.
## Logical Operations
### Actions
| Action | SSL primitive | Evidence |
|--------|---------------|----------|
| Validate brief + mode | `VALIDATE` | Clarification protocol, mode inference |
| Select provider strategy | `SELECT` | Vendor matrix, `providers.*.order`, availability |
| Read brief / capture | `READ` | Brief text, `--capture` path |
| Generate script | `CALL_TOOL` | AgentScriptProvider -> `script.json` |
| Synthesize narration + timing | `CALL_TOOL` | oma-voice -> `audio/*.wav` + `timing.json` |
| Produce visuals | `CALL_TOOL` | oma-image / oma-slide / stock -> `visuals/*` |
| Build captions | `WRITE` | key-free `captions.srt` / `.vtt` from timing |
| Compose render-spec | `WRITE` | `render-spec.json` (determinism boundary) |
| Render video | `CALL_TOOL` | Remotion / MPT compositor -> `<mode>-<slug>.mp4` |
| Validate result | `VALIDATE` | Schema parse, manifest hashes, exit code |
| Report output | `NOTIFY` | Run-dir + mp4 path summary |
### Tools and instruments
- `oma video generate`, `oma video doctor`, `oma video list-providers`, `oma video render`
- Provider adapters: AgentScript, oma-voice, oma-image, oma-slide, Pexels, Pixelle, oma-captions, Cap, Remotion, MPT
- Remotion authoring specs (`resources/remotion-authoring/`), prompt tips, vendor matrix, video config
### Canonical command path
```bash
oma video doctor
oma video generate "<brief>" --mode shorts --aspect auto --captions tiktok --format json
```
Explainer from a README, with a deterministic seed:
```bash
oma video generate "explain this project" --mode explainer --aspect 16:9 --seed 42 --out ./out
```
Demo from a screen capture:
```bash
oma video generate "feature walkthrough" --mode demo --capture <absolute-path>.mp4
```
Deterministic re-render from an existing run:
```bash
oma video render .agents/results/videos/20260603-143052-ab12cd-shorts
```
### Resource scope
| Scope | Resource target |
|-------|-----------------|
| `LOCAL_FS` | Briefs, captures, assets, render-spec, run dir, manifests |
| `PROCESS` | oma-image / oma-slide CLIs, Remotion / MPT, Cap CLI, Browser web-capture driver (subprocess) |
| `NETWORK` | Voicebox MCP (localhost), oma-image vendor APIs, the user-supplied `--url` for web capture (masked in logs/manifest), optional Pexels / Pixelle / RunningHub |
| `CREDENTIALS` | oma-image vendor auth, optional `PEXELS_API_KEY` / `RUNNINGHUB_API_KEY`. Web capture handles NO credentials — a human logs in if the flow needs it; nothing is stored or printed. |
### Preconditions
- Brief carries enough signal for the mode, or the user approves the inferred/amplified plan.
- Output path is inside `$PWD` (or `--allow-external-out` is set).
- For `demo` `--source file`: the capture path exists, is absolute/$PWD-guarded, and is a valid format.
- - For `demo` `--source web`: a `--url` is supplied (else `SchemaValidationError`); an available browser capture runtime + an interactive TTY exist (else the run falls back to the guided protocol).
+ - For `demo --source web`, supply `--url` as context and `--capture <path>` as recording input; without a recording the command returns guided capture instructions.
- Required provider availability holds for the chosen (non-fallback) path, or the fallback is acceptable.
### Effects and side effects
- Creates a run directory with assets, render-spec, captions, mp4, and manifest.
- oma-voice plays narration on the speakers as a side effect of synthesis.
- May call paid or rate-limited providers (Pexels / Pixelle / RunningHub) only when keys are present.
### Guardrails
1. **Clarify or infer before invoking**: if the mode/topic is ambiguous, infer the mode from keywords and show the user the plan, or ask. Do NOT silently render from a vague brief. See `Clarification Protocol` below.
2. **Key-optional dispatch (backend rule 11)**: every external capability has a real (key/resource) path AND a key-free fallback. Paid providers (Pexels, Pixelle) auto-enable only when their env key is present; otherwise the chain falls through to oma-image stills + Ken Burns. Default providers (oma-voice local, oma-image, oma-slide, Remotion) are key-free, so auto-triggering on keywords is safe.
3. **Cost guardrail**: confirm before runs whose estimated cost is >= `cost.guardrail_usd` (`$0.20`, configurable) or `--max-usd`. `--yes` / `OMA_VIDEO_YES=1` bypass. Local/free paths carry zero cost.
4. **Path safety**: output paths outside `$PWD` require `--allow-external-out`. `--capture` is absolutized, `$PWD`-guarded, and format-validated; external assets are copied into the run dir and hashed (no URL refs).
5. **Cancellable**: SIGINT/SIGTERM aborts in-flight provider calls, the render, and the orchestrator.
6. **Deterministic outputs**: `render-spec.json` + asset files (+ seed + embedded Pretendard font) are the determinism boundary. `oma video doctor --install` fetches the Pretendard woff2 once; if offline, the render gracefully falls back to system fonts, and byte-identical output across machines is only guaranteed once the font is present. Re-rendering the same render-spec is byte-stable; `OMA_VIDEO_MOCK=1` replays golden fixtures.
7. **Limits**: `limits.max_duration_sec` = 180, `limits.max_scenes` = 40 (wall-time + memory bound).
8. **Community-MCP consent**: Pixelle-MCP is off by default and requires one-time explicit consent + source review before connecting; RunningHub credits gate on `--max-usd`.
- 9. **Demo is human-in-the-loop**: capture is performed by a human. For `--source file` the skill guides but does not screen-record autonomously; for `--source web` the tool only opens a headed browser and records while the human drives the entire on-screen flow (interactive ENTER to stop). The mechanism prescribes nothing about what the flow is or what the recording is for.
- 10. **Web-capture security**: NO credential automation of any kind — if a flow needs a login, a human performs it. The driver runs as a subprocess under the browser capture runtime (never imported into the CLI). The `--url` and any query tokens are masked in logs and in `manifest.json`; credentials are never stored or printed. Recording and all outputs are confined to the run dir. On-screen sensitive input is captured as-is — the user controls the flow. Multi-page navigation (popup / new tab / redirect) is recorded generically, with no assumption about the flow's shape.
- 11. **Web capture is key-optional + non-blocking**: web capture is the real branch; the guided protocol is the fallback when the browser capture runtime is unavailable OR there is no interactive TTY (CI / `-y` / no stdin) — the run falls back to guided and never hangs. Live capture is outside the determinism boundary, so the manifest records `nondeterministic: true`.
- 12. **Headed capture needs a display**: web capture launches a **headed** Chromium so the human can drive the flow. On display-less hosts (CI / Linux without X), pass `--capture-stop duration:<sec>|selector:<css>` — the driver then runs headless — or expect a capture error / guided fallback.
+ 9. **Demo is human-in-the-loop**: capture is performed by a human. Both source modes ingest a supplied recording or return guided recording instructions.
+ 10. **Capture privacy**: a human controls recording and login. Keep URL query tokens masked in logs and manifests; validate supplied recordings before ingestion.
+ 11. **Guided capture is non-blocking**: when no recording is supplied, return capture-required instructions without opening a browser or waiting for stdin.
+ 12. **Recording input**: supply a human-recorded video with `--capture <path>`; use Cap or another available screen recorder.
13. **Run-dir retention**: `.agents/results/videos/<run>/` accumulates one directory per run (assets + mp4 + manifest) and is **never auto-pruned**; the user deletes old run directories manually.
14. **Exit codes align with `oma search fetch`** (0 ok, 1 generic, 2 safety, 3 not-found, 4 invalid-input, 5 auth-required, 6 timeout).
### Clarification Protocol
Before invoking `oma video generate`, the calling agent runs this checklist. **If any answer is "no / unknown", clarify or infer-and-confirm with the user first.**
**Required signal (must be present or inferable):**
- [ ] **Mode**: shorts / explainer / demo? Infer from keywords (shorts/reels/쇼츠/릴스 -> shorts; README/code/data/explain/설명 -> explainer; demo/walkthrough/capture/데모 -> demo).
- [ ] **Topic / source**: what is the video about? (a topic, a README/code path, a capture file, or — for `demo` `--source web` — a `--url`)
- [ ] **For `demo`**: `--source file` (a `--capture` path) or `--source web` (a `--url`)? For `--source web`, state up front that a **human drives the on-screen flow** and that the tool **never automates any login**.
**Strongly recommended (ask if absent AND not inferable):**
- [ ] **Aspect**: `9:16` (shorts/reels), `16:9` (explainer/demo), `1:1`, or `auto` (snaps to the mode default).
- [ ] **Locale**: narration + caption language (default from config; translated via oma-translation when non-source).
- [ ] **Captions**: `tiktok` (centered, static windowed cues), `lower-third`, or `none`.
- [ ] **Duration**: target seconds (<= 180) or `auto` (derived from the script).
- [ ] **Voice / music**: voice profile or `none`; music `upbeat` / `calm` / `none`. **The default voice is `none` → a silent video with estimated caption timing.** Pass `--voice <profile>` (a Voicebox profile) whenever narration is expected. Music is rendered offline by Strudel and mixed at −18 dB; it needs a one-time `oma video doctor --install-strudel` and degrades to no music without it.
**Amplification shortcut.** For a one-line brief (e.g. "shorts about Jeju coffee"), do not pop a questionnaire if the request is genuinely simple. Instead **amplify inline and show the user** the inferred plan before invoking:
> User: "make a short about Jeju coffee"
> Agent: "I'll generate this as: *mode `shorts`, 9:16, ~30s, oma-image stills with Ken Burns, TikTok captions, locale `en`, calm music*. Proceed, or adjust mode/aspect/voice?"
Skip clarification when the user authored a full brief (mode + topic + aspect + captions). Respect their flags verbatim.
**Output language.** Narration and on-screen text are authored in the requested locale. Image-generation prompts passed to oma-image are sent in English (image models are trained predominantly on English captions); translate the user's request and show the translated version during amplification.
### Modes
| Mode | Aspect | Source | Default visual | Compositor | Output |
|------|:---:|--------|----------------|------------|--------|
| `shorts` | 9:16 | synthetic (topic) | oma-image stills + Ken Burns; Pexels (key) · Pixelle AIGC (key) opt | Remotion · MPT alt | `shorts-<slug>.mp4` |
| `explainer` | 16:9 / 9:16 | README · code · data | oma-slide frames + oma-image diagrams + code | Remotion (deterministic) | `explainer-<slug>.mp4` |
- | `demo` | 16:9 | `--source file` (Cap / capture file) · `--source web` (headed browser at `--url`) | raw footage (default) · Remotion intro · zoom · callouts (`--polish`) | Remotion polish | `demo-<slug>.mp4` |
+ | `demo` | 16:9 | `--source file` (Cap / capture file) · `--source web` (URL context + guided capture) | raw footage (default) · Remotion intro · zoom · callouts (`--polish`) | Remotion polish | `demo-<slug>.mp4` |
### 3-Tier Integration
| Tier | Surface | Providers | Trigger |
|:---:|---------|-----------|---------|
| 1 | CLI-first (subprocess, deterministic) | Remotion render, MPT, oma-image, oma-slide, oma-voice | always available (key-free defaults) |
| 2 | MCP | Voicebox MCP (voice/timing), Pixelle-MCP (AIGC, off by default) | MCP server reachable; Pixelle needs explicit consent + key |
- | 3 | Guided (human-in-the-loop) | Headed web capture (`--source web`, human drives the flow), Cap (capture), guided protocol fallback | `demo` mode; web capture needs an available browser capture runtime + a TTY (else guided protocol) |
+ | 3 | Guided (human-in-the-loop) | Cap recording and guided capture with `--capture <path>` | `demo` mode; supply a recording or follow the capture instructions |
### Invocation
#### Standalone
```
/oma-video make a 30s short about Jeju coffee
/oma-video --mode explainer --aspect 16:9 explain this project from the README
/oma-video --mode demo --source file --capture ~/recordings/walkthrough.mp4 feature demo
/oma-video --mode demo --source web --url http://localhost:3000 record my app flow
/oma-video --mode demo --source web --url <url> --ready-selector "#app" --polish onboarding clip
```
#### Shell CLI
```
oma video generate "<brief>" [--mode shorts|explainer|demo] \
[--aspect 9:16|16:9|1:1|auto] [--locale <lang>] \
[--captions tiktok|lower-third|none] \
[--visual auto|generate|stock|aigc|slide] \
[--voice <profile>|none] [--music upbeat|calm|cinematic|lofi|piano|none] \
[--duration <sec>|auto] [--compositor remotion|mpt] \
[--capture <path>] \
[--source file|web] [--url <url>] [--device <name>] \
[--ready-selector <css>] [--show-cursor] [--polish] \
[--capture-timeout <sec>] [--capture-stop duration:<sec>|selector:<css>] \
[--out <dir>] [--allow-external-out] \
[--max-usd <n>] [--seed <n>] [--timeout 600] [-y] \
[--dry-run] [--script <path>] \
[--format text|json] [--no-brief-in-manifest]
# --script: inject the agent-authored script.json (agent-as-key). Without it the
# CLI builds its own skeleton script from the brief — always pass the script
# the agent wrote so narration/on-screen text/visual prompts are honored.
- # --source web: headed browser at --url; capture size derived from --aspect/--device (no hardcoded size).
+ # Both source modes use guided capture; pass --capture <path> to ingest a recording.
# A human drives the on-screen flow; press ENTER to stop. NO credential automation. --url/tokens masked.
- # Non-interactive (CI / -y / no TTY) or an unavailable browser capture runtime -> falls back to the guided protocol (no hang).
- # --capture-stop gives CI a non-interactive stop (duration / selector) in place of the ENTER prompt.
+ # Both source modes use guided capture; pass --capture <path> to ingest a recording.
+ # Browser recording controls do not start an automatic recorder; capture the flow separately.
oma video doctor # readiness report only (no install): Node/Chromium/FFmpeg · Remotion toolchain · remotion-dev/skills · Pretendard font · MPT · Voicebox MCP · oma-image vendors · Pixelle-MCP · Cap
oma video doctor --install # warm the latest Remotion toolchain (deps + Chrome Headless Shell + Pretendard) and remotion-dev/skills into ~/.cache/oma-video
oma video doctor --upgrade # force a latest check now
oma video compose <runDir> [--format json] # scaffold/refresh the run's Remotion project + print the authoring contract
oma video doctor --install-mpt # one-time: MoneyPrinterTurbo checkout (clone + venv + deps) for --compositor mpt
oma video list-providers # availability + key/fallback status
oma video render <runDir> # re-render from render-spec.json (deterministic)
```
#### Shared Infrastructure (from other skills)
Other skills call `oma video generate --format json` and parse the JSON envelope (`{exitCode, runDir, manifestPath, scriptPath, renderSpecPath, warnings, error}`) from stdout. There is no `outputs` key — read output/asset paths from the manifest at `manifestPath`. The deterministic boundary is `render-spec.json` + assets, so a downstream consumer can re-render via `oma video render <runDir>` without re-running script/voice/visual generation.
### Output Layout
```
.agents/results/videos/
└── 20260603-143052-ab12cd-shorts/ # {timestamp}-{shortid}-{mode}
├── script.json # determinism boundary start
├── timing.json
├── render-spec.json # deterministic compute boundary
├── audio/
│ └── narration-01.wav # single narration track (ALL lines joined — not per-scene)
├── visuals/
│ └── scene-01.jpg …
├── captions.srt
├── captions.vtt
├── shorts-<slug>.mp4
└── manifest.json # reproducibility record
```
### Audio Sync & Captions Format
- **Narration is one wav**: oma-voice joins every scene line into a single `audio/narration-01.wav`, referenced by `render-spec.audio.narration`. There are no per-scene `narration-NN.wav` files.
- **Timing**: per-line offsets live in `timing.json` (voicebox-stt -> estimated; the `tts-native` and `whisper-cpp` source values are reserved but not yet wired — `TODO(oma-deferred): whisper-cpp`); scene boundaries and caption cues are derived from it.
- **Captions**: key-free `.srt` (+ `.vtt`) built from `timing.json`; `render-spec.captions.file` points at the `.srt`. The compositor renders **static windowed cues** — the cue active at the current frame, CSS-wrapped (no per-word animation).
- **Music**: `--music <preset>` renders a BGM bed with **Strudel** and mixes it under narration at `render-spec.audio.musicGainDb` (default −18 dB). The bed is generated offline (headless Chrome + `OfflineAudioContext`), so it needs no key, no network, and no audio device — a 30s bed renders in well under a second.
- **Presets**: `calm` (sustained pad + arpeggio), `upbeat` (bright plucks), `cinematic` (drone build to a lead), `lofi` (warm chords, swung ticks), `piano` (neoclassical arpeggio). Each preset picks its key and mode from the run `seed`, so the same preset sounds different run to run without a second pattern.
- **Artifacts** in the run dir: `music/bgm.wav` (mixed by the compositor), `music/bgm.mp3` (preview), `music/bgm-raw.wav` (pre-master), and `music/pattern.strudel` — the source that produced them, editable and re-renderable by hand.
- **Level**: every bed is normalised to −14 LUFS with a static gain before a peak limiter, so `musicGainDb` means the same thing for every preset. Normalisation is deliberately *not* `loudnorm`'s one-pass mode, which flattens the arrangement arc.
- **Opt-in install**: `@strudel/*` is AGPL-3.0-or-later while the oma CLI is MIT, so the deps are never bundled and never installed implicitly. Run `oma video doctor --install-strudel` once. The CLI never imports Strudel — it spawns `resources/strudel/render.mjs` as a subprocess, the same boundary the Remotion / web-capture projects use.
- **Fallback**: a missing install, a missing Chrome, or a failed render degrades to *no music* with a warning. The run still succeeds and `audio.music` stays unset (never a dangling `staticFile()` ref).
- **Determinism**: the built-in beds are oscillator-only (sine / triangle / square / sawtooth), which render byte-identically on replay. Noise sounds (`white` / `pink` / `brown`) draw from `Math.random()` and would break that, so the templates avoid them.
## References
Follow `resources/execution-protocol.md` step by step.
See `resources/vendor-matrix.md` for provider precheck + fallback-chain rules.
Author `--script` files against `resources/script-schema.md` (full field reference + example; `schemaVersion: "1.0"` is required).
Use `resources/prompt-tips.md` for writing effective briefs per mode.
Before submitting, run `resources/checklist.md`.
Remotion compositions are agent-authored per run — see `resources/remotion-authoring/README.md`.
The MPT fallback compositor driver lives at `resources/mpt/driver.py` (consumed by the CLI's mpt-project internals).
### Configuration
Project-specific settings: the `video:` section of `.agents/oma-config.yaml`, which `oma update` preserves. Shipped defaults live in the CLI (`DEFAULT_VIDEO_CONFIG` in `cli/commands/video/config.ts`) — write only the keys you change. The legacy `config/video-config.yaml` is no longer read by the CLI; migration 022 moves anything you had changed there into oma-config (and deletes the file when it was never edited).
Env vars: `OMA_VIDEO_DEFAULT_MODE`, `OMA_VIDEO_DEFAULT_OUT`, `OMA_VIDEO_YES`, `PEXELS_API_KEY`, `RUNNINGHUB_API_KEY` (+ `POLLINATIONS_API_KEY` via oma-image), `OMA_VIDEO_MOCK`.
- Execution steps: `resources/execution-protocol.md`
- Vendor matrix: `resources/vendor-matrix.md`
- Prompt tips: `resources/prompt-tips.md`
- Checklist: `resources/checklist.md`
- Remotion authoring: `resources/remotion-authoring/README.md` (+ `shorts.md`, `explainer.md`, `demo.md`)
- Context loading: `../_shared/core/context-loading.md`