voiceover · diff

git:20261006.ecf1c91 to git:20261007.4c3f477

121 added, 29 removed. Audit A to A.

---
name: voiceover
- description: Write and produce voiceover in BashCut — prepare the text so text-to-speech reads it right, synthesize several takes with a voice.synthesize provider (for example VieNeu-TTS for Vietnamese, preset or cloned voices), check every take, place it on the Voiceover layer and fit it to the picture. Use for narration, voiceover, fixing a flubbed line without re-recording, or when the user says "lồng tiếng", "thuyết minh", "đọc lời", "giọng AI", "clone giọng", "text to speech", "TTS".
+ description: Write and produce voiceover in BashCut — measure the voice's speaking rate and the speech-free windows, size each line to its window, prepare the text so text-to-speech reads it right, synthesize takes with a voice.synthesize provider (for example VieNeu-TTS for Vietnamese, preset or cloned voices), check every take against its text, fit it to the picture, re-take one line in place, and dub or localise existing speech. Use for narration, voiceover, fixing a flubbed line without re-recording, dubbing, or when the user says "lồng tiếng", "thuyết minh", "đọc lời", "giọng AI", "clone giọng", "text to speech", "TTS", "dịch lồng tiếng", "lồng tiếng tiếng Anh".
---
# Voiceover
- Reply in the user's language.
+ Reply in the user's language. Commands measure and act; the numbers below are ranges with their source, to
+ choose from with the measurements in hand. Measure → change → measure again.
- ## Provider
+ ## Provider and voice
- `voice speak` uses the project's `voice.synthesize` provider. Check with `plugins list`; if there is none,
- `plugins search --capability voice.synthesize` and ask the user to install one (for Vietnamese: VieNeu TTS).
- Voice choice, cloning from a reference file and variation are the plugin's options:
+ `voice speak` uses the project's `voice.synthesize` provider. Check with `capabilities get voice.synthesize`:
+ `reason: missing` → `plugins search --capability voice.synthesize` and ask the user to install one (for Vietnamese:
+ VieNeu TTS); `not_configured` → tell the user what the provider's `detail` says (turn it on, trust it again);
+ `unhealthy` → the failing dependency is in `detail`.
```sh
+ bashcut voice voices # per voice: language, region, style, gender, supportsRate, measuredRate; per provider: clones
bashcut plugins options bashcut.vieneu-tts
bashcut plugins option bashcut.vieneu-tts --option voice --value <voice>
```
- Only clone the user's own voice or a voice whose owner agreed. If viewers could take the AI voice for a real
- recording, suggest disclosing it.
+ - Pick voices by their facts (language, region, style, measured rate), then audition 2–3 on the **hardest** line,
+ not the first; the first voice is rarely the pick (T10 §2).
+ - Cloning: only the user's own voice or one whose owner agreed. A provider that `clones` refuses without
+ `--clone-consent`; pass it only when the user said yes in this conversation, never by default (T10 §6). If
+ viewers could take the AI voice for a real recording, suggest disclosing it.
- ## 1. Prepare the text
+ ## 1. Measure before writing
+ There is no normal speaking rate: it varies by voice, language, genre and energy (T10 §3). Measure it in the
+ content language's unit (syllables for Vietnamese, characters for Chinese/Japanese/Korean, else words):
+
+ ```sh
+ bashcut speech rate # each speaker in the transcribed footage: p10/p50/p90, overall, articulation
+ bashcut speech rate --voice PROVIDER/VOICE # the rates measured on that voice's earlier takes
+ ```
+
+ - A voice with no measured rate yet: synthesize one test line (`--keep-takes`) and read its `unitsPerSecond`.
+ - When the voiceover should sound like the creator, size lines from the creator's own measured rate.
+ - Never reuse an English words-per-second figure for a syllable language (T10 §3).
+
+ Then find where voiceover can go and how much text fits:
+
+ ```sh
+ bashcut narration windows --min-seconds S --rate R # speech-free windows; with rate, a unit budget each
+ ```
+
+ Each window comes with what covers it (music, footage, silence), the shots under it and anchors (last word
+ before it, first cut, first beat). Write each line to its window's budget; the p10–p90 spread of the voice tells
+ you how much slack to leave. If a line does not fit, **cut text** first; do not plan to speed the voice
+ (T02 §4). TTS length drifts from any plan (a 50 s script came out 40–45 s, T10 §3), so the final timing comes
+ from the generated take, not from the script.
+
+ ## 2. Prepare the text
+
Spell out what gets misread:
- - Numbers, years, times: `2026` → "hai nghìn không trăm hai mươi sáu", `7h30` → "bảy giờ ba mươi".
+ - Numbers, years, times: `2026` → "hai nghìn không trăm hai mươi sáu", `7h30` → "bảy giờ ba mươi". Captions
+ keep the digits (T10 §2).
- Units and symbols: km, %, °C, `k` (nghìn đồng).
- Acronyms as letters ("xê en xê" for CNC); show them as letters again in captions.
- English or unusual words: rephrase in plain words rather than respelling ("chill" was read "chiều").
- Very short exclamations (≤ 5 words) with an expressive voice fail often; use a calm voice or a longer line.
When a phrase is misread in every take, **rewording beats regenerating** ("kín xe" → "chật cứng xe"). Change
approved text minimally and tell the user which words changed.
- ## 2. Synthesize
+ ## 3. Synthesize
```sh
- bashcut voice speak "Text of one line" --takes 3 --at-frame F # job; inserts the best take on Voiceover
- bashcut voice speak "Text" --takes 6 --keep-takes # keep every take, insert nothing
- bashcut jobs status JOB_ID # chosen take, its score and all take scores
+ bashcut voice speak "Text of one line" --takes 3 --at-frame F --target-rate R # job; inserts the take closest to R
+ bashcut voice speak "Text" --takes 6 --keep-takes # keep every take, insert nothing
+ bashcut jobs wait JOB_ID --timeout 25 # repeat until done; every take's facts and file
```
- - One call per line or short paragraph, placed where it belongs; it becomes one undoable edit.
- - Generation is random: 3 takes for normal lines, 5–6 for important or difficult ones.
- - With `--keep-takes`, pick a take and place it with `media import /abs/take.wav --kind audio --place --track
- VOICEOVER_TRACK --at-frame F`.
+ - Each take reports `seconds`, `units`, `unitsPerSecond`, `leadingSilence`, `trailingSilence`, `pauses` and its
+ file. Choose by `--target-rate` (the rate you sized the line with), `--choose N` (a take you picked from the
+ facts), or else the provider's score. Core holds no pace formula: the choice is yours.
+ - One call per line or short paragraph at its window; it becomes one undoable edit and the item keeps its text.
+ - Give every call a stable `--request-id` (`vo-<section>-<n>`). A paid provider: `--dry-run` first and show the
+ user its `estimate`; the job's `usage` reports what it charged.
+ - Takes: 3–6, more for short or expressive lines (T10 §3).
+ - A take whose rate is far from the voice's measured rate (video-recap flags beyond about ±30 %, T10 §3) has
+ likely dropped, repeated or invented words: check it first.
+ - Style or emotion directives: compare with a neutral take of the same text. One source treats +25 % duration as
+ "took effect" and < 5 % as "not applied" (jianshuo, T10 §3); listen as well.
+ - With `--keep-takes`, place a take with `media import /abs/take.wav --kind audio --place --track VOICEOVER_TRACK
+ --at-frame F --base-rev N`.
- ## 3. Check every kept take
+ ## 4. Check every kept take
- The model sometimes misreads, drops or repeats words, mostly **at the start** of a line. A text match score
- barely drops for repeated words. Transcribe the kept takes (`captions generate --media TAKE_MEDIA`, or listen)
- and compare with the text; regenerate or reword failures. Recognition also mishears some regional accents
- (s→x, final n→ng): count only errors a listener would hear.
+ ```sh
+ bashcut voice check --item ITEM # job: similarity, words matched/substituted/missing, extra words
+ bashcut voice check --media MEDIA --text "The text it should say"
+ ```
- ## 4. Fit to picture
+ The model misreads, drops or repeats words, mostly **at the start** of a line. Read `words` and `extra`, not
+ only `similarity`: a repeated word barely moves it. Pass `--min-similarity` only when you want a pass/fail flag
+ (one kit gates at 0.92 for Chinese, T10 §3); the call is yours. Recognition also mishears some regional accents
+ (s→x, final n→ng): count only errors a listener would hear. Fix by rewording or a re-take.
- - Never over real speech; keep ≥ 0.3 s from real voices (`bc:audio-mix`).
- - Give each voiceover section b-roll that is longer than the line; a too-short source clip made the voiceover
- run into the next spoken cut.
- - Captions for voiceover: write them from your text (`bc:captions-text`).
- - Level ~−16 LUFS with music 12–18 dB under (ducking on the music layer).
+ ## 5. Fit to picture
+
+ Fix in order of naturalness (T10 §4, jianshuo ladder):
+
+ 1. **Shorten or reword the text**, then re-take.
+ 2. **Provider rate**, when the voice `supportsRate` (jianshuo slows −12…−15 % before stretching, T10 §3).
+ 3. **Stretch with pitch kept**, within bounds you pass:
+ ```sh
+ bashcut voice fit --item ITEM --to-frame F --min-ratio MIN --max-ratio MAX --base-rev N
+ ```
+ Outside the bounds nothing changes and the error gives the speed it would need. Ranges seen: slowing to
+ 0.82–0.95 (floors 0.82 jianshuo, 0.85 digitalsamba; imperceptible above ~0.92); speeding up to about 1.1 and
+ only after rewriting failed (T10 §3). How much stretch goes unheard varies with the voice's timbre and how
+ much music masks it (T10 §3).
+ 4. **Change the text more**, with the user's consent (it changes the captions).
+
+ Silence at a line end is often better than a stretched voice. Ranges seen: tail after the last word 0.1–0.5 s,
+ longer before a cut or reveal; gaps between sentences 0.2–0.8 s, up to 1–3 s after a key line (T10 §3).
+
+ - Never over real speech. Keep a gap from real voices (0.3 s worked in our edits); set the project's
+ `review.voiceoverMarginSeconds` so `review run` checks it.
+ - Give each voiceover section b-roll longer than the line: a too-short source clip made the voiceover run into the
+ next spoken cut.
+ - Captions for voiceover: write them from your text (`bc:captions-text`); `voice speak` times the item's captions
+ from the take.
+ - Level: judged against the music, not one number. Measure both (`audio measure --media ID`: loudness and
+ `presenceShare`) and set the balance with `bc:audio-mix`, which holds the ranges and sources.
+
+ ## 6. Re-take one line
+
+ ```sh
+ bashcut voice speak --replace ITEM --takes 3 --target-rate R # same text, new take in place
+ bashcut voice speak "Reworded line" --replace ITEM --takes 3 # new text
+ ```
+
+ The new take keeps the item's place and its captions are timed again. Check it (`voice check --item ITEM`) and
+ fit it again if its length changed.
+
+ ## 7. Dubbing and localisation
+
+ 1. **Transcribe** the original (`media transcribe --media ID`, then `media transcript --media ID`) and measure
+ both rates: the original speaker (`speech rate --media ID`) and the target voice.
+ 2. **Translate** per cue, keeping cue boundaries at sentence ends. Languages differ in length (Mandarin takes
+ 60–80 % of the time Spanish does, jianshuo, T10 §2), so size each cue from the target voice's measured rate.
+ 3. **Sample one cue first**: the most performance-sensitive one, not the first. Get the user's yes on voice and
+ pace before batching (OpenMontage, T10 §2).
+ 4. **Per cue, fit ladder**: shorten the text → re-take → `voice fit` within bounds → move the cue
+ (`timeline move ITEM --track TRACK --at-frame F --base-rev N`) into nearby silence. Tell the user which cues
+ were shortened (it changes the meaning) and which moved.
+ 5. **Keep the original audio as a bed** under the dub, not muted: jianshuo keeps it at 0.15–0.25 linear
+ (≈ −16…−12 dB, T10 §3); set it with `volumeDb` and check with `bc:audio-mix`.
+ 6. Check every cue (`voice check`), then re-time captions from the takes.