oma-video · diff
git:20260905.43b3fc7 to git:20260909.e50712e
112 added, 292 removed. Audit A to A.
---
name: oma-video
- description: Short-form, explainer, and demo video generation via a key-optional
- 3-tier router. Composes scripts, oma-voice narration,
- oma-image/oma-slide/stock visuals, key-free captions, and a per-run
- agent-authored Remotion composition (always-latest Remotion +
- remotion-dev/skills) into reproducible run directories. Routes three modes —
- shorts/reels (9:16), explainer (16:9 README/code/data), and demo/walkthrough
- (screen capture from human-recorded video). Use for video, shorts, reels,
- short-form, demo, explainer, walkthrough, screencast, web capture, video
+ description: Create short-form, explainer, and human-recorded demo videos
+ through the oma video CLI. Routes script, narration, visuals, captions, and an
+ agent-authored Remotion composition into a reproducible run directory. Use for
+ video, shorts, reels, demo, explainer, walkthrough, screencast, video
generation, 영상, 숏폼, 쇼츠, 릴스, 데모, 설명 영상.
---
- # Video Agent - Short-form, Explainer & Demo Router
+ # Video Router
## Scheduling
- ### Goal
- Generate finished `.mp4` videos through a key-optional, 3-tier (CLI-first / MCP / guided) provider router while preserving deterministic asset buses (script -> timing -> render-spec), reproducible manifests, cost controls, and capture-path safety.
-
- ### Intent signature
- - User asks for a short-form video, shorts/reels clip, TikTok/YouTube Short, explainer, demo, walkthrough, or screencast.
- - User wants a topic, README, code, or data turned into a narrated, captioned video.
- - Another skill needs shared video-generation infrastructure (script -> assets -> render).
-
### When to use
- - Generating short-form video (shorts / reels) from a topic or brief (`--mode shorts`, 9:16)
- - Generating an explainer from a README, code, or data set (`--mode explainer`, 16:9 / 9:16)
- - Producing a demo / walkthrough from a screen capture file (`--mode demo --source file`, 16:9)
- - Guided screen capture for demos, walkthroughs, onboarding clips, bug reproductions, and app reviews. Supply the recording with `--capture <path>`; `--source web --url <url>` supplies URL context.
- - Re-rendering an existing run deterministically from `render-spec.json`
- - Other skills needing video-generation infrastructure (shared invocation via `--format json`)
-
- ### When NOT to use
-
- - Generating a single still image -> use `oma-image`
- - Generating a slide deck / presentation -> use `oma-slide` (this skill calls it internally for explainer frames)
- - Generating speech audio only (no video) -> use `oma-voice`
- - Non-linear video editing of an existing finished mp4 -> out of scope (OpenCut-MCP deferred)
- - Human-recorded web flows are in scope (`--source web`); live streaming is out of scope
- - Interactive HTML explainer document (not a video) -> use `oma-explanation`
+ Use this skill for a short/reel, README or code explainer, or demo walkthrough.
- ### Expected inputs
- - A brief (topic / README path / data) plus optional mode, aspect, locale, captions, visual, voice, music, duration, compositor, capture path, seed
- - For `demo` `--source file`: a screen-capture file path (`--capture`) or Cap availability
- - For `demo --source web`, supply `--url` as context and `--capture <path>` as recording input; without a recording the command returns guided capture instructions.
- - Authentication/environment state for oma-voice (Voicebox MCP), oma-image vendors, and optional Pexels / Pixelle keys
+ | Request | Mode | Default aspect | Required source |
+ |---|---|---|---|
+ | Short, reel, social clip | `shorts` | `9:16` | Topic or brief |
+ | README, code, data explanation | `explainer` | `16:9` | Topic or source path |
+ | Demo or walkthrough | `demo` | `16:9` | Human recording via `--capture` |
- ### Expected outputs
- - A run directory under `.agents/results/videos/<timestamp>-<shortid>-<mode>/`
- - Deterministic asset bus: `script.json`, `timing.json`, `render-spec.json`
- - `audio/`, `visuals/`, `captions.srt` / `captions.vtt`, the rendered `<mode>-<slug>.mp4`
- - `manifest.json` with providers, asset hashes, cost breakdown, and exit code
+ ### When NOT to use
- ```yaml
- outputs:
- - name: video
- description: rendered <mode>-<slug>.mp4 in the run directory
- artifact: ".agents/results/videos/*/*.mp4"
- required: true
- - name: manifest
- description: reproducibility record (providers, asset sha256 hashes, cost breakdown, exit code)
- artifact: ".agents/results/videos/*/manifest.json"
- required: true
- - name: render-spec
- description: deterministic compute boundary consumed by `oma video render <runDir>`
- artifact: ".agents/results/videos/*/render-spec.json"
- required: true
- - name: captions
- description: key-free caption tracks aligned to timing.json
- artifact: ".agents/results/videos/*/captions.srt"
- required: false
- ```
+ Use `oma-image` for a still image, `oma-slide` for a deck, `oma-voice` for
+ audio only, and `oma-explanation` for an interactive HTML explainer. Editing an
+ existing finished video and live streaming are out of scope.
- ### Dependencies
- - `oma video generate` CLI + central error module (exit codes aligned with `oma search fetch`)
- - oma-voice (Voicebox MCP), oma-image, oma-slide as key-free fallback providers
- - Per-run Remotion project at `<runDir>/remotion/` scaffolded by `oma video compose` on the always-latest toolchain (`~/.cache/oma-video/remotion/<ver>/`); the composition is agent-authored per run following remotion-dev/skills (`resources/remotion-authoring/`)
- - `resources/vendor-matrix.md`, `resources/execution-protocol.md`, `resources/prompt-tips.md`, and the `video:` section of `.agents/oma-config.yaml`
+ ## Structural Flow
- ### Control-flow features
- - Branches by mode (shorts / explainer / demo), aspect, visual strategy, provider availability, cost threshold, capture requirement, and path safety
- - Runs a per-capability fallback chain (real key/resource -> key-free fallback) per backend rule 11
- - Reads briefs/captures and writes assets, render-spec, and manifests
- - Calls external resources: Voicebox MCP, oma-image vendors, Remotion toolchain, optional Pexels / Pixelle
+ Inputs are a brief plus optional mode, aspect, locale, captions, visual, voice,
+ music, duration, compositor, capture path, and seed. Outputs live in
+ `.agents/results/videos/<timestamp>-<shortid>-<mode>/`:
- ## Structural Flow
+ - `script.json`, `timing.json`, and `render-spec.json` form the deterministic
+ asset bus.
+ - Captions and acquired audio/visual assets are recorded in `manifest.json`
+ with hashes, providers, cost, warnings, and exit code.
+ - A successful real render contains `<mode>-<slug>.mp4`, an encoded video
+ stream, and a positive ffprobe duration.
- ### Entry
- 1. Validate that the brief carries enough mode/topic signal (or infer the mode from keywords).
- 2. For `demo`, confirm a capture path exists (or Cap is available); otherwise enter the guided protocol.
- 3. Resolve defaults from shipped code defaults -> the `video:` section of `.agents/oma-config.yaml` -> env vars -> CLI flags; check output path safety and limits.
+ `OMA_VIDEO_MOCK=1` is a test harness only. It can create deterministic text
+ placeholders with an `.mp4` name; those files are never a user deliverable. A
+ missing Remotion/MPT toolchain, an un-authored composition, a render error, or
+ an invalid video fails with diagnostics and leaves the script/render spec for
+ recovery.
- ### Scenes
- 1. **PREPARE**: Resolve mode/aspect/locale, clarify or amplify the brief, choose the visual + compositor strategy.
- 2. **ACQUIRE**: Probe provider availability (voice / visual / caption / compositor), validate capture path, check cost.
- 3. **ACT**: Run the mode pipeline — script -> (voice ∥ visuals ∥ captions) -> render-spec -> compositor render.
- 4. **VERIFY**: Validate every asset-bus schema, manifest hashes, exit code, and the output mp4.
- 5. **FINALIZE**: Return the run-dir path, the mp4 path, and any provider/coverage warnings.
+ ### Decide and confirm
- ### Transitions
- - If the brief lacks a clear mode, infer from keywords (shorts/reels -> shorts; README/code -> explainer; capture -> demo) and show the user the inferred plan before generating.
- - If the selected visual provider key is absent (Pexels / Pixelle), fall through the chain to the key-free oma-image stills + Ken Burns fallback and annotate coverage.
- - For `demo`, ingest `--capture <path>` or provide guided capture instructions. `--source web` requires `--url` as context; it does not start a browser recorder.
- - If `demo` `--source file` has no capture and Cap is unavailable, emit the guided capture protocol and stop (exit code maps to capture-required).
- - If estimated cost (Pixelle / RunningHub credits) exceeds the guardrail, require confirmation unless bypassed.
+ Infer mode when clear: short/reel -> `shorts`; README/code/data/explain ->
+ `explainer`; demo/walkthrough/capture -> `demo`. For a one-line request, state
+ the inferred mode, aspect, duration, visual strategy, captions, locale, and
+ voice/music before invoking. Do not make the user complete a questionnaire
+ when those defaults are clear.
- ### Failure and recovery
- - If a provider is unavailable, try the next provider in the capability's `order`; only chain exhaustion is a stage failure.
- - Remotion is always the latest npm release and oma owns NO composition code: `oma video compose <runDir>` refreshes the toolchain + remotion-dev/skills (throttled) and scaffolds `<runDir>/remotion/`; you author `src/Root.tsx` per its `AUTHORING.md` and `resources/remotion-authoring/<mode>.md`, then `oma video render <runDir>`. A tsc/render failure is a composition bug: re-read the skills, fix, re-render (no fixed cap; stop only after two consecutive attempts without progress and report).
- - If the toolchain cannot be fetched (offline, nothing cached), point the user to `oma video doctor --install` once online; fall back to the MPT compositor where applicable (MPT itself needs a one-time `oma video doctor --install-mpt`).
- - If Voicebox MCP is down, fall back to estimated timing (still produces captions). A whisper.cpp hop between the two is reserved but not yet wired (`TODO(oma-deferred): whisper-cpp`).
- - If the brief locale is non-source, translate via oma-translation (key-free); if absent, warn and keep source text.
+ Ask only when it changes the result: an ambiguous mode/source, a required demo
+ recording, or a cost confirmation. Respect an explicit mode, aspect, duration,
+ captions, or voice verbatim.
- ### Exit
- - Success: `<mode>-<slug>.mp4` and `manifest.json` exist in the run directory; all schemas validate.
- - Partial success: video renders with a key-free fallback in place of a paid provider; coverage is annotated in warnings.
- - Failure: no video is produced and the route/cost/capture/auth/safety blocker is explicit in the exit code + manifest.
+ For a demo, a human records the screen and controls login. `--source web --url`
+ provides context only; it never automates login or starts a recorder. Without
+ `--capture`, return guided capture instructions and stop.
## Logical Operations
- ### Actions
- | Action | SSL primitive | Evidence |
- |--------|---------------|----------|
- | Validate brief + mode | `VALIDATE` | Clarification protocol, mode inference |
- | Select provider strategy | `SELECT` | Vendor matrix, `providers.*.order`, availability |
- | Read brief / capture | `READ` | Brief text, `--capture` path |
- | Generate script | `CALL_TOOL` | AgentScriptProvider -> `script.json` |
- | Synthesize narration + timing | `CALL_TOOL` | oma-voice -> `audio/*.wav` + `timing.json` |
- | Produce visuals | `CALL_TOOL` | oma-image / oma-slide / stock -> `visuals/*` |
- | Build captions | `WRITE` | key-free `captions.srt` / `.vtt` from timing |
- | Compose render-spec | `WRITE` | `render-spec.json` (determinism boundary) |
- | Render video | `CALL_TOOL` | Remotion / MPT compositor -> `<mode>-<slug>.mp4` |
- | Validate result | `VALIDATE` | Schema parse, manifest hashes, exit code |
- | Report output | `NOTIFY` | Run-dir + mp4 path summary |
+ ### Guardrails
- ### Tools and instruments
- - `oma video generate`, `oma video doctor`, `oma video provider list`, `oma video render`
- - Provider adapters: AgentScript, oma-voice, oma-image, oma-slide, Pexels, Pixelle, oma-captions, Cap, Remotion, MPT
- - Remotion authoring specs (`resources/remotion-authoring/`), prompt tips, vendor matrix, video config
+ 1. Keep output and capture paths inside `$PWD` unless external output is
+ explicitly allowed. Validate capture formats and copy external assets into
+ the run directory. Mask URL query/hash tokens in logs and manifests.
+ 2. Provider configuration is key-optional: use the configured chain. Paid
+ providers require their environment key and the cost guardrail. Local
+ fallbacks may replace voice, visuals, captions, or music; record coverage in
+ warnings. A compositor failure is never a fallback video.
+ 3. Confirm estimated spend at or above `cost.guardrail_usd` or `--max-usd`
+ unless `--yes` or `OMA_VIDEO_YES=1` authorizes it.
+ 4. Respect `limits.max_duration_sec` (180) and `limits.max_scenes` (40).
+ Cancel subprocess work on SIGINT/SIGTERM.
+ 5. Keep run directories. Never auto-prune a user’s video artifacts.
+ 6. `--dry-run` writes only planning artifacts and does no provider render.
+ It does not prove an MP4 exists.
### Canonical command path
- ```bash
- oma video doctor
- oma video generate "<brief>" --mode shorts --aspect auto --captions tiktok --output json
- ```
- Explainer from a README, with a deterministic seed:
```bash
- oma video generate "explain this project" --mode explainer --aspect 16:9 --seed 42 --output-dir ./out
- ```
-
- Demo from a screen capture:
- ```bash
- oma video generate "feature walkthrough" --mode demo --capture <absolute-path>.mp4
- ```
-
- Deterministic re-render from an existing run:
- ```bash
- oma video render .agents/results/videos/20260603-143052-ab12cd-shorts
- ```
-
- ### Resource scope
- | Scope | Resource target |
- |-------|-----------------|
- | `LOCAL_FS` | Briefs, captures, assets, render-spec, run dir, manifests |
- | `PROCESS` | oma-image / oma-slide CLIs, Remotion / MPT, Cap CLI, Browser web-capture driver (subprocess) |
- | `NETWORK` | Voicebox MCP (localhost), oma-image vendor APIs, the user-supplied `--url` for web capture (masked in logs/manifest), optional Pexels / Pixelle / RunningHub |
- | `CREDENTIALS` | oma-image vendor auth, optional `PEXELS_API_KEY` / `RUNNINGHUB_API_KEY`. Web capture handles NO credentials — a human logs in if the flow needs it; nothing is stored or printed. |
-
- ### Preconditions
- - Brief carries enough signal for the mode, or the user approves the inferred/amplified plan.
- - Output path is inside `$PWD` (or `--allow-external-out` is set).
- - For `demo` `--source file`: the capture path exists, is absolute/$PWD-guarded, and is a valid format.
- - For `demo --source web`, supply `--url` as context and `--capture <path>` as recording input; without a recording the command returns guided capture instructions.
- - Required provider availability holds for the chosen (non-fallback) path, or the fallback is acceptable.
-
- ### Effects and side effects
- - Creates a run directory with assets, render-spec, captions, mp4, and manifest.
- - oma-voice plays narration on the speakers as a side effect of synthesis.
- - May call paid or rate-limited providers (Pexels / Pixelle / RunningHub) only when keys are present.
-
- ### Guardrails
-
- 1. **Clarify or infer before invoking**: if the mode/topic is ambiguous, infer the mode from keywords and show the user the plan, or ask. Do NOT silently render from a vague brief. See `Clarification Protocol` below.
- 2. **Key-optional dispatch (backend rule 11)**: every external capability has a real (key/resource) path AND a key-free fallback. Paid providers (Pexels, Pixelle) auto-enable only when their env key is present; otherwise the chain falls through to oma-image stills + Ken Burns. Default providers (oma-voice local, oma-image, oma-slide, Remotion) are key-free, so auto-triggering on keywords is safe.
- 3. **Cost guardrail**: confirm before runs whose estimated cost is >= `cost.guardrail_usd` (`$0.20`, configurable) or `--max-usd`. `--yes` / `OMA_VIDEO_YES=1` bypass. Local/free paths carry zero cost.
- 4. **Path safety**: output paths outside `$PWD` require `--allow-external-out`. `--capture` is absolutized, `$PWD`-guarded, and format-validated; external assets are copied into the run dir and hashed (no URL refs).
- 5. **Cancellable**: SIGINT/SIGTERM aborts in-flight provider calls, the render, and the orchestrator.
- 6. **Deterministic outputs**: `render-spec.json` + asset files (+ seed + embedded Pretendard font) are the determinism boundary. `oma video doctor --install` fetches the Pretendard woff2 once; if offline, the render gracefully falls back to system fonts, and byte-identical output across machines is only guaranteed once the font is present. Re-rendering the same render-spec is byte-stable; `OMA_VIDEO_MOCK=1` replays golden fixtures.
- 7. **Limits**: `limits.max_duration_sec` = 180, `limits.max_scenes` = 40 (wall-time + memory bound).
- 8. **Community-MCP consent**: Pixelle-MCP is off by default and requires one-time explicit consent + source review before connecting; RunningHub credits gate on `--max-usd`.
- 9. **Demo is human-in-the-loop**: capture is performed by a human. Both source modes ingest a supplied recording or return guided recording instructions.
- 10. **Capture privacy**: a human controls recording and login. Keep URL query tokens masked in logs and manifests; validate supplied recordings before ingestion.
- 11. **Guided capture is non-blocking**: when no recording is supplied, return capture-required instructions without opening a browser or waiting for stdin.
- 12. **Recording input**: supply a human-recorded video with `--capture <path>`; use Cap or another available screen recorder.
- 13. **Run-dir retention**: `.agents/results/videos/<run>/` accumulates one directory per run (assets + mp4 + manifest) and is **never auto-pruned**; the user deletes old run directories manually.
- 14. **Exit codes align with `oma search fetch`** (0 ok, 1 generic, 2 safety, 3 not-found, 4 invalid-input, 5 auth-required, 6 timeout).
-
- ### Clarification Protocol
-
- Before invoking `oma video generate`, the calling agent runs this checklist. **If any answer is "no / unknown", clarify or infer-and-confirm with the user first.**
+ # Plan or create the asset bus. Supply --script whenever an agent authored it.
+ oma video generate "Jeju coffee" --mode shorts --aspect 9:16 \
+ --captions tiktok --script ./script.json --output json
- **Required signal (must be present or inferable):**
- - [ ] **Mode**: shorts / explainer / demo? Infer from keywords (shorts/reels/쇼츠/릴스 -> shorts; README/code/data/explain/설명 -> explainer; demo/walkthrough/capture/데모 -> demo).
- - [ ] **Topic / source**: what is the video about? (a topic, a README/code path, a capture file, or — for `demo` `--source web` — a `--url`)
- - [ ] **For `demo`**: `--source file` (a `--capture` path) or `--source web` (a `--url`)? For `--source web`, state up front that a **human drives the on-screen flow** and that the tool **never automates any login**.
+ # Deterministic planning only; no real render or provider work.
+ oma video generate "explain this project" --mode explainer --seed 42 --dry-run
- **Strongly recommended (ask if absent AND not inferable):**
- - [ ] **Aspect**: `9:16` (shorts/reels), `16:9` (explainer/demo), `1:1`, or `auto` (snaps to the mode default).
- - [ ] **Locale**: narration + caption language (default from config; translated via oma-translation when non-source).
- - [ ] **Captions**: `tiktok` (centered, static windowed cues), `lower-third`, or `none`.
- - [ ] **Duration**: target seconds (<= 180) or `auto` (derived from the script).
- - [ ] **Voice / music**: voice profile or `none`; music `upbeat` / `calm` / `none`. **The default voice is `none` → a silent video with estimated caption timing.** Pass `--voice <profile>` (a Voicebox profile) whenever narration is expected. Music is rendered offline by Strudel and mixed at −18 dB; it needs a one-time `oma video doctor --install-strudel` and degrades to no music without it.
+ # Human-recorded demo input.
+ oma video generate "feature walkthrough" --mode demo --capture <absolute-path>.mp4
- **Amplification shortcut.** For a one-line brief (e.g. "shorts about Jeju coffee"), do not pop a questionnaire if the request is genuinely simple. Instead **amplify inline and show the user** the inferred plan before invoking:
+ # Scaffold the per-run Remotion project, author src/Root.tsx as instructed,
+ # then render and validate the encoded output.
+ oma video compose <runDir> --output json
+ oma video render <runDir> --output json
- > User: "make a short about Jeju coffee"
- > Agent: "I'll generate this as: *mode `shorts`, 9:16, ~30s, oma-image stills with Ken Burns, TikTok captions, locale `en`, calm music*. Proceed, or adjust mode/aspect/voice?"
+ # Diagnose required toolchains without changing a run.
+ oma video doctor
+ oma video provider list
+ ```
- Skip clarification when the user authored a full brief (mode + topic + aspect + captions). Respect their flags verbatim.
+ `oma video generate --output json` returns
+ `{exitCode, runDir, manifestPath, scriptPath, renderSpecPath, warnings, error}`.
+ Read video and asset paths from the manifest; the JSON envelope has no
+ `outputs` field.
- **Output language.** Narration and on-screen text are authored in the requested locale. Image-generation prompts passed to oma-image are sent in English (image models are trained predominantly on English captions); translate the user's request and show the translated version during amplification.
+ ### Failure and recovery
- ### Modes
+ - Missing Remotion composition: `oma video compose <runDir>` prepares the
+ project and authoring contract. Author `<runDir>/remotion/src/Root.tsx` using
+ the generated `AUTHORING.md`, then invoke `oma video render <runDir>`. The
+ command typechecks, renders, and ffprobes the output. Fix a reported composition
+ or toolchain failure and re-run; return a failure report when it cannot render.
- | Mode | Aspect | Source | Default visual | Compositor | Output |
- |------|:---:|--------|----------------|------------|--------|
- | `shorts` | 9:16 | synthetic (topic) | oma-image stills + Ken Burns; Pexels (key) · Pixelle AIGC (key) opt | Remotion · MPT alt | `shorts-<slug>.mp4` |
- | `explainer` | 16:9 / 9:16 | README · code · data | oma-slide frames + oma-image diagrams + code | Remotion (deterministic) | `explainer-<slug>.mp4` |
- | `demo` | 16:9 | `--source file` (Cap / capture file) · `--source web` (URL context + guided capture) | raw footage (default) · Remotion intro · zoom · callouts (`--polish`) | Remotion polish | `demo-<slug>.mp4` |
+ - Missing MPT toolchain: `--compositor mpt` requires the installed checkout, its virtual environment,
+ and ffmpeg. Use `oma video doctor --install-mpt` when setup is authorized and
+ available. MPT setup failures, driver failures, and non-video output fail with
+ the diagnostic; they do not write a placeholder MP4.
- ### 3-Tier Integration
+ Success means all asset schemas and manifest hashes validate and a real MP4
+ passes video-stream and duration validation. A partial success may use a
+ key-free visual, timing, caption, or music fallback, but it still requires that
+ real video validation.
- | Tier | Surface | Providers | Trigger |
- |:---:|---------|-----------|---------|
- | 1 | CLI-first (subprocess, deterministic) | Remotion render, MPT, oma-image, oma-slide, oma-voice | always available (key-free defaults) |
- | 2 | MCP | Voicebox MCP (voice/timing), Pixelle-MCP (AIGC, off by default) | MCP server reachable; Pixelle needs explicit consent + key |
- | 3 | Guided (human-in-the-loop) | Cap recording and guided capture with `--capture <path>` | `demo` mode; supply a recording or follow the capture instructions |
+ ## References
- ### Invocation
+ ### Conditional resources
- #### Standalone
- ```
- /oma-video make a 30s short about Jeju coffee
- /oma-video --mode explainer --aspect 16:9 explain this project from the README
- /oma-video --mode demo --source file --capture ~/recordings/walkthrough.mp4 feature demo
- /oma-video --mode demo --source web --url http://localhost:3000 record my app flow
- /oma-video --mode demo --source web --url <url> --ready-selector "#app" --polish onboarding clip
- ```
+ Load only what the task needs:
- #### Shell CLI
- ```
- oma video generate "<brief>" [--mode shorts|explainer|demo] \
- [--aspect 9:16|16:9|1:1|auto] [--locale <lang>] \
- [--captions tiktok|lower-third|none] \
- [--visual auto|generate|stock|aigc|slide] \
- [--voice <profile>|none] [--music upbeat|calm|cinematic|lofi|piano|none] \
- [--duration <sec>|auto] [--compositor remotion|mpt] \
- [--capture <path>] \
- [--source file|web] [--url <url>] [--device <name>] \
- [--ready-selector <css>] [--show-cursor] [--polish] \
- [--capture-timeout <sec>] [--capture-stop duration:<sec>|selector:<css>] \
- [--output-dir <dir>] [--allow-external-output] \
- [--max-usd <n>] [--seed <n>] [--timeout 600] [-y] \
- [--dry-run] [--script <path>] \
- [--output text|json] [--no-brief-in-manifest]
- # --script: inject the agent-authored script.json (agent-as-key). Without it the
- # CLI builds its own skeleton script from the brief — always pass the script
- # the agent wrote so narration/on-screen text/visual prompts are honored.
- # Both source modes use guided capture; pass --capture <path> to ingest a recording.
- # A human drives the on-screen flow; press ENTER to stop. NO credential automation. --url/tokens masked.
- # Both source modes use guided capture; pass --capture <path> to ingest a recording.
- # Browser recording controls do not start an automatic recorder; capture the flow separately.
- oma video doctor # readiness report only (no install): Node/Chromium/FFmpeg · Remotion toolchain · remotion-dev/skills · Pretendard font · MPT · Voicebox MCP · oma-image vendors · Pixelle-MCP · Cap
- oma video doctor --install # warm the latest Remotion toolchain (deps + Chrome Headless Shell + Pretendard) and remotion-dev/skills into ~/.cache/oma-video
- oma video doctor --upgrade # force a latest check now
- oma video compose <runDir> [--output json] # scaffold/refresh the run's Remotion project + print the authoring contract
- oma video doctor --install-mpt # one-time: MoneyPrinterTurbo checkout (clone + venv + deps) for --compositor mpt
- oma video provider list # availability + key/fallback status
- oma video render <runDir> # re-render from render-spec.json (deterministic)
- ```
+ - `resources/execution-protocol.md` for the full ordered pipeline, failure
+ mapping, and JSON reporting rules.
+ - `resources/vendor-matrix.md` before changing providers, keys, cost, or
+ fallback order.
+ - `resources/script-schema.md` when authoring or validating `--script` input.
+ - `resources/prompt-tips.md` when turning a brief into scene prompts.
+ - `resources/remotion-authoring/README.md` and the selected mode guide before
+ writing `Root.tsx`.
+ - `resources/checklist.md` before handing a real video to a user.
- #### Shared Infrastructure (from other skills)
- Other skills call `oma video generate --output json` and parse the JSON envelope (`{exitCode, runDir, manifestPath, scriptPath, renderSpecPath, warnings, error}`) from stdout. There is no `outputs` key — read output/asset paths from the manifest at `manifestPath`. The deterministic boundary is `render-spec.json` + assets, so a downstream consumer can re-render via `oma video render <runDir>` without re-running script/voice/visual generation.
+ ### Verification
- ### Output Layout
+ For CLI/runtime changes, add a regression test for the affected success and
+ failure paths. At minimum run the focused Vitest files, for example:
- ```
- .agents/results/videos/
- └── 20260603-143052-ab12cd-shorts/ # {timestamp}-{shortid}-{mode}
- ├── script.json # determinism boundary start
- ├── timing.json
- ├── render-spec.json # deterministic compute boundary
- ├── audio/
- │ └── narration-01.wav # single narration track (ALL lines joined — not per-scene)
- ├── visuals/
- │ └── scene-01.jpg …
- ├── captions.srt
- ├── captions.vtt
- ├── shorts-<slug>.mp4
- └── manifest.json # reproducibility record
+ ```bash
+ cd cli
+ bunx vitest run commands/video/providers/compositor.test.ts \
+ commands/video/orchestrator.test.ts
+ bunx biome check commands/video/providers/compositor.ts \
+ commands/video/providers/compositor.test.ts
```
- ### Audio Sync & Captions Format
-
- - **Narration is one wav**: oma-voice joins every scene line into a single `audio/narration-01.wav`, referenced by `render-spec.audio.narration`. There are no per-scene `narration-NN.wav` files.
- - **Timing**: per-line offsets live in `timing.json` (voicebox-stt -> estimated; the `tts-native` and `whisper-cpp` source values are reserved but not yet wired — `TODO(oma-deferred): whisper-cpp`); scene boundaries and caption cues are derived from it.
- - **Captions**: key-free `.srt` (+ `.vtt`) built from `timing.json`; `render-spec.captions.file` points at the `.srt`. The compositor renders **static windowed cues** — the cue active at the current frame, CSS-wrapped (no per-word animation).
- - **Music**: `--music <preset>` renders a BGM bed with **Strudel** and mixes it under narration at `render-spec.audio.musicGainDb` (default −18 dB). The bed is generated offline (headless Chrome + `OfflineAudioContext`), so it needs no key, no network, and no audio device — a 30s bed renders in well under a second.
- - **Presets**: `calm` (sustained pad + arpeggio), `upbeat` (bright plucks), `cinematic` (drone build to a lead), `lofi` (warm chords, swung ticks), `piano` (neoclassical arpeggio). Each preset picks its key and mode from the run `seed`, so the same preset sounds different run to run without a second pattern.
- - **Artifacts** in the run dir: `music/bgm.wav` (mixed by the compositor), `music/bgm.mp3` (preview), `music/bgm-raw.wav` (pre-master), and `music/pattern.strudel` — the source that produced them, editable and re-renderable by hand.
- - **Level**: every bed is normalised to −14 LUFS with a static gain before a peak limiter, so `musicGainDb` means the same thing for every preset. Normalisation is deliberately *not* `loudnorm`'s one-pass mode, which flattens the arrangement arc.
- - **Opt-in install**: `@strudel/*` is AGPL-3.0-or-later while the oma CLI is MIT, so the deps are never bundled and never installed implicitly. Run `oma video doctor --install-strudel` once. The CLI never imports Strudel — it spawns `resources/strudel/render.mjs` as a subprocess, the same boundary the Remotion / web-capture projects use.
- - **Fallback**: a missing install, a missing Chrome, or a failed render degrades to *no music* with a warning. The run still succeeds and `audio.music` stays unset (never a dangling `staticFile()` ref).
- - **Determinism**: the built-in beds are oscillator-only (sine / triangle / square / sawtooth), which render byte-identically on replay. Noise sounds (`white` / `pink` / `brown`) draw from `Math.random()` and would break that, so the templates avoid them.
-
- ## References
-
- Follow `resources/execution-protocol.md` step by step.
- See `resources/vendor-matrix.md` for provider precheck + fallback-chain rules.
- Author `--script` files against `resources/script-schema.md` (full field reference + example; `schemaVersion: "1.0"` is required).
- Use `resources/prompt-tips.md` for writing effective briefs per mode.
- Before submitting, run `resources/checklist.md`.
- Remotion compositions are agent-authored per run — see `resources/remotion-authoring/README.md`.
- The MPT fallback compositor driver lives at `resources/mpt/driver.py` (consumed by the CLI's mpt-project internals).
-
- ### Configuration
-
- Project-specific settings: the `video:` section of `.agents/oma-config.yaml`, which `oma update` preserves. Shipped defaults live in the CLI (`DEFAULT_VIDEO_CONFIG` in `cli/commands/video/config.ts`) — write only the keys you change. The legacy `config/video-config.yaml` is no longer read by the CLI; migration 022 moves anything you had changed there into oma-config (and deletes the file when it was never edited).
- Env vars: `OMA_VIDEO_DEFAULT_MODE`, `OMA_VIDEO_DEFAULT_OUT`, `OMA_VIDEO_YES`, `PEXELS_API_KEY`, `RUNNINGHUB_API_KEY` (+ `POLLINATIONS_API_KEY` via oma-image), `OMA_VIDEO_MOCK`.
-
- - Execution steps: `resources/execution-protocol.md`
- - Vendor matrix: `resources/vendor-matrix.md`
- - Prompt tips: `resources/prompt-tips.md`
- - Checklist: `resources/checklist.md`
- - Remotion authoring: `resources/remotion-authoring/README.md` (+ `shorts.md`, `explainer.md`, `demo.md`)
- - Context loading: `../_shared/core/context-loading.md`
+ Do not run a live render merely to test documentation or a mock-only branch.