ai-video-generate git:20261013.ec9e4cbA

Immutable. This exact content never changes and is served at /api/v1/blob/0ab96796a5663bc2.

---
name: ai-video-generate
description: "Generation orchestration for approved ai_video storyboards: dispatch mode selection, scene generation via kling_generate, audio assembly, regenQueue processing. Load after storyboard approval."
step: true
---

# AI Video Generation

You are executing generation for an approved `ai_video` storyboard. The workflow file `workflows/ai_video.json` lists this as the `generate` step, following `ai-video-plan`. The storyboard — `scenes[]`, `imageRefs[]`, `styleAnchor` — was written by the planning skill; your job is to generate clips and audio from it.

This skill covers **Phase 6** (scene generation dispatch) and **Phase 7** (draft-phase regenQueue processing). For Phases 0-2 (story clarification, storyboard writes, review), see `skills/ai-video-plan/SKILL.md`.

## Sub-skills

| Name | Path | When to load |
|------|------|--------------|
| `eval-scenes` | `skills/eval-scenes/SKILL.md` | After generating scenes in Phase 6 — quality evaluation + retry loop |

---

## Status machine

```
pending → storyboard_ready → draft → final
```

| Transition | Writer | Precondition |
|------------|--------|--------------|
| `pending → storyboard_ready` | **ai-video-plan** (agent) | Phase 1 writes complete: imageRefs anchors+images filled, styleAnchor written, scenes[] populated. `tracks[0].items` still `[]`. |
| `storyboard_ready → storyboard_ready` | **ai-video-plan** (agent) in review-phase | User asks for storyboard changes via chat. Mutate `project.json`; status stays `storyboard_ready`. |
| `(implicit, UI-driven)` | **UI** | User clicks Approve. UI writes `storyboard.approval = {approvedAt: <ISO8601>}`. Status stays `storyboard_ready` — you observe the field and start Phase 6. |
| `storyboard_ready → draft` | **you** (this skill) | Every `storyboard.scenes[i]` has a matching clip in `tracks[0].items` (either `generation.sceneId === scene.id` OR a `batchShots[]` entry for it). |
| `draft → final` | **human** (via UI) | Manual review. Out of scope for this skill. |

---

## Vendor

Each scene generates through one of three vendors: **Kling** (`kling_generate`, default), **Seedance** (`seedance_generate`, on fal.ai) or **Kinovi** (`kinovi_generate`: Seedance and Wan models on one Kinovi key). Kling is project-aware (Steps B/C/D below); Seedance and Kinovi are not, so the mechanics differ enough to cover separately.

**Picking the vendor.** Kling is the default for every scene. Generate a scene with Seedance instead when the storyboard or that scene sets `vendor: "seedance"`, with Kinovi when it sets `vendor: "kinovi"`, or with either when the user asks for it by name ("use Kinovi for the drone shot"). Never pick a vendor the user didn't ask for, and use one video vendor for the whole video unless the storyboard says otherwise.

**Calling Seedance.** `seedance_generate` is standalone only, with no `--project-id`/`--scene-id` mode. Compose the prompt yourself first (same composition as Step C: styleAnchor prefix + scene prose + camera tags), then call:

```
seedance_generate --prompt <composed> --out <path> \
  [--image <first_frame> [--end-image <last_frame>]] [--ref-image <path> ...] \
  [--duration <n or auto>] [--sound]
```

- **Modes**, picked by which image flags you pass (mutually exclusive with each other):
  - Text only: text-to-video.
  - `--image` (optionally `--end-image`): image-to-video. Its **aspect ratio always follows the first frame**, so don't pass `--aspect-ratio`.
  - `--ref-image` (repeatable): reference-to-video. The prompt refers to them as `@Image1`, `@Image2`… in list order, not Kling's `<<<image_N>>>` tokens.
- **Duration:** the default model (`seedance-2.5`) takes `4`-`30` seconds, or `"auto"`.
- **Sound is off unless you pass `--sound`**: it costs more, and Seedance's own default is on.
- Seedance has no chained or batched mode. A scene routed to Seedance always generates independently, even when the rest of the storyboard is running chained or batched.
- Append the clip to `tracks[0].items` and save `project.json` yourself, the same shape Step D uses for a Kling clip, but with `generation.provider: "fal"` and `generation.model` set to the Seedance model actually used (e.g. `seedance-2.5`). `seedance_generate` doesn't touch the project.

**Calling Kinovi.** `kinovi_generate` is standalone too. Compose the prompt the same way, then:

```
kinovi_generate --prompt <composed> --out <path> [--model seedance2-5] \
  [--image <first_frame> [--end-image <last_frame>]] [--ref-image <path> ...] \
  [--ref-video <path> ...] [--ref-audio <path> ...] [--duration <n>] [--sound] [--json]
```

- **Models:** `seedance2-5` (default, 4-30 s, up to 1080p), `seedance-20` (4-15 s, up to 4k), `seedance2-fast` and `seedance2.0-mini` (cheaper, 480p/720p), `wan3.0-text-to-video` and `wan3.0-image-to-video` (2-30 s). Wan refuses anything that looks like a real person's face.
- **Modes** follow the inputs: text only; `--image` (and `--end-image`) for frames, whose aspect ratio follows the first frame; `--ref-image`/`--ref-video`/`--ref-audio` for Seedance reference mode (the prompt names them @Image1, @Video1, @Audio1… in list order). Local files are uploaded to Kinovi for you.
- **Sound is off unless you pass `--sound`.**
- **Cost:** credits are taken when the task starts and refunded if it fails. A cheap look first: `--draft` (`seedance2-5`, `--resolution 480p`), then `--upgrade <draft taskId>` for the 1080p version.
- **A run that times out** prints the task id. Rerun with `--task-id <id> --out <path>` to collect it: never generate the scene again, which would pay twice.
- Save the clip like a Seedance one, with `generation.provider: "kinovi"` and `generation.model` the model used.

**On failure:** a `ConnectorError` with reason `invalid_api_key` or `insufficient_credit`, from any vendor, means the integration itself needs fixing, not the prompt. Stop, tell the user which one to fix in Integrations (Kling, Seedance/fal.ai or Kinovi), and **never fall back to another vendor silently**. A scene the user asked to generate with Seedance failing over to Kling (or any other swap) changes what they asked for without telling them. A `content_refused` failure (moderation, a real person's face, a copyrighted reference) is the prompt or the reference, not the key: change it and tell the user what you changed.

---

## Phase 6 — Scene generation (status: `storyboard_ready`, `storyboard.approval` is set)

### Entry point — how you get here

You are not file-watching. You enter Phase 6 because **the user tells you to** in chat. The typical message looks like:

> *"I approved the storyboard for project "X". Please proceed with scene generation per the ai-video-generate skill Phase 6 contract."*

That message is produced by either:
- The UI's post-approval "Tell your agent" panel (a one-click "Copy" button on `ApproveAndGenerate`), or
- The `montaj approve` CLI command (which prints the same message for the user to paste), or
- The user typing it manually (e.g. they edited `project.json` directly and now want you to go).

### Verification guard — **always run before generating**

Before running the `kling_generate` step, re-read `project.json` and verify:

1. `project.projectType === "ai_video"`.
2. `project.status === "storyboard_ready"`.
3. `project.storyboard.approval` exists AND has an `approvedAt` field.
4. `project.storyboard.scenes` is non-empty.

If ANY of those is false, **do not proceed.** Tell the user what's missing and ask them to click the Approve button in the UI (or run `montaj approve`). Common failure modes:

- User said "I approved, go" but `storyboard.approval` is absent → the UI save failed, or the user saw a stale view. Ask them to click Approve again.
- Status is `pending` → storyboard hasn't been written yet. Return to Phase 1.
- Status is already `draft` → nothing to do. Tell them.
- `scenes` is empty → Phase 1 never populated the plan. Return to Phase 1 instead of generating from thin air.

Never trust the user's chat message alone as the authorization signal. The file field is the source of truth — the message is just the trigger to check.

### Once the guard passes, start generating.

### Step A — determine the scene set to generate

Iterate `storyboard.scenes[]`. For each scene, check for a matching clip on `tracks[0].items`:

- **Single-shot origin**: `tracks[0].items.some(c => c.generation?.sceneId === scene.id)`.
- **Batched origin**: `tracks[0].items.some(c => c.generation?.batchShots?.some(shot => shot.sceneId === scene.id))`.

A scene is "already done" if EITHER check passes. Then:

- Clip exists → **skip** (user is re-approving after editing only a subset).
- `scene.lastError` set AND no clip → **retry**.
- Else → **new generation**.

This makes generation idempotent and incremental:

- **Full approval (first run):** all scenes generate.
- **Re-approval after editing specific scenes:** only the changed scenes regenerate. Unchanged scenes keep their existing clips.
- **Selective regeneration (user asks to redo specific scenes):** remove the clip for that scene from `tracks[0].items`, then run the `kling_generate` step with `--project-id X --scene-id Y`. The step's dedup guard replaces any existing clip for the same sceneId, so you can also just re-run the step: it overwrites.

**Never regenerate all scenes when only some changed.** Each Kling call costs credits and takes ~60s. If the user edited scenes 1 and 3, generate only those two. The agent should diff the user's changes against the current storyboard to determine which scenes need regeneration.

### Step B — pick a dispatch mode

Three modes; infer from the prompt + scene count (ask in Phase 0 if unclear):

#### Independent (default — parallel)

Each scene is a separate run of the `kling_generate` step, self-contained. **You MUST start these in parallel**: start every scene's `kling_generate` run in a single response so they execute concurrently. Cap at 4 concurrent calls; if there are more than 4 scenes, fire 4 at a time, wait for any to complete, then fire the next. Do NOT generate scenes one at a time in a sequential loop, which wastes minutes of wall-clock time that parallel dispatch avoids.

When parallel results land out of narrative order, compute each clip's `start`/`end` from the scene's position in `storyboard.scenes[]`, not from generation order. Fault-isolated and easy to regenerate.

**How to fire in parallel:** Include multiple tool calls in one assistant message. For example, if you have 5 scenes and are capping at 4 concurrent:

1. First message: start `kling_generate` for scenes 1, 2, 3, 4 at once (4 calls in one response).
2. When results arrive, write all 4 clips to `tracks[0].items`.
3. Second message: start `kling_generate` for scene 5.
4. Write the final clip and check if all scenes are done → set status to `draft`.

#### Chained continuity

Scene N visually continues N-1. Generate sequentially: after N-1 succeeds, run the `snapshot` step with `--input <clip.src> --at <clip.outPoint> --out <frame.jpg>` to extract its last frame, then run the `kling_generate` step with `--first-frame <frame.jpg> ...` for scene N. Slower (no parallelism), but preserves visual continuity at scene boundaries.

#### Batched (multi-shot)

Group up to 6 scenes into ONE `kling_generate` call using `--multi-shot --shot-type customize --multi-prompt '<json>'`. Kling returns a single concatenated video. Cheaper (1 billing unit per batch vs N), stronger visual consistency across shots (shared stylistic context), one poll instead of N.

Trade-offs:
- **Per-shot prompt cap is 512 chars**, not 2500. Write tighter per-scene prose.
- **No `--first-frame` / `--last-frame`** — frame control isn't supported. Incompatible with chained.
- **All-or-nothing failure.** If the batch fails on one scene, the whole batch is lost. Regenerating a single scene from a completed batch runs as a single-shot call spliced into `tracks[0].items`.
- **One clip per batch, not per scene.** Store per-scene mapping in `generation.batchShots[]` (see Step D).
- **Storyboard > 6 scenes** → split into multiple batches.

#### Picking a mode

- **Default for nearly everything → independent (parallel).** Fastest wall-clock time, fault-isolated, no prompt-length restrictions. Use this unless one of the below specifically applies.
- "Flowing" / "seamless" transitions requiring frame continuity (e.g. a continuous camera move across scenes) → **chained**. Only use when the creative brief specifically demands visual continuity at scene boundaries.
- Strong scene-to-scene continuity **AND** scenes fit in 512 chars each **AND** you want cheapest billing → **batched**. Best cost-to-quality when applicable.

State your chosen mode in chat once at the start — the user can redirect if wrong.

#### Picking a model

Two Kling models are available. Pass `--model <name>` to `kling_generate`.

- **`kling-v3-omni`** (default and best model): flexible 3-15s durations, multi-shot support, start+end frame in both std/pro. **Generates audio** when `sound: "on"`. Use this unless a scene specifically needs o1.
- **`kling-video-o1`** (earlier model): **Only 5s or 10s durations.** No multi-shot. End frame (`--last-frame`) requires `--mode pro`. **Does NOT generate audio**, clips are silent.

**How to decide:** The model is **per-scene, not per-project**, so you can mix and match, but the step never switches models on its own. Stay on `kling-v3-omni` unless a scene needs o1's look and can do without audio; then pass `--model kling-video-o1` explicitly. Kling generates in `--mode pro` by default (the higher-quality of std/pro).

The connector snaps invalid durations to the nearest allowed value, but snapping changes your editorial pacing, so pick the right model and duration per scene rather than rely on snapping.

State your model choices in chat alongside the dispatch mode. Record the actual model used on `generation.model` for each clip.

#### Transition style — hard cuts by default

Each scene is generated from scratch via text-to-video. Shot-to-shot transitions are **hard cuts**. Use `--first-frame` (chained mode) only when the user explicitly asks for visual continuity at a scene boundary.

Identity consistency across hard cuts comes from **the same character specs and ref images being passed to every scene** — not from pixel handoff between frames. The `kling_generate` step appends character descriptions and `<<<image_N>>>` tokens to every prompt, so identity holds via the spec.

### Step C — prompt composition (handled by the step)

In project-aware mode (`--project-id` + `--scene-id`), the `kling_generate` step handles all prompt composition automatically. The agent does NOT compose the wire prompt manually. The step:

1. Reads the scene's `## Section` prompt and flattens it (Subject → Action → Dialogue → Setting order, period-terminated).
2. Prepends `storyboard.styleAnchor` as a short prefix.
3. Places `<<<image_N>>>` tokens inline at character/object label mentions (matches on first word of label for flexibility — "Rosie" matches even if label is "Rosie the Dog").
4. Appends `[SHOT SCALE]` and `[CAMERA MOVE]` tags from the scene's structured fields.
5. Auto-sets a default negative prompt targeting common Kling failure modes.
6. Generates a random seed for reproducibility.

**The agent's only job is writing good `## Section` prompts in Phase 1.** Everything else is mechanical.

**What gets stored on `generation.prompt`:** the composed wire-ready string (with tokens, style anchor, camera tags). This is the exact prompt sent to Kling.

- **Length caps (enforced by connector):**
  - **Single-shot: silently truncates at 2500 chars.** Keep scene prompts under ~100 words to stay safe.
  - **Multi-shot customize: hard-rejects any `multi_prompt[i].prompt` > 512 chars.**

- Resolve `scene.refImages` IDs against `storyboard.imageRefs` (use `imageRefs[i].refImages[0]` as the primary path). Enforce the API cap of 7 refs per scene.

- Respect Kling's length cap: 2500 chars in single-shot, **512 chars per shot in multi-shot**.

### Step D — call and write

#### Single-shot (independent or chained)

```
kling_generate \
  --prompt <combined> \
  --duration <scene.duration> \
  --aspect-ratio <storyboard.aspectRatio> \
  --model <chosen model> \
  --ref-image <path> [--ref-image <path> ...] \
  --out <path> \
  --external-task-id <scene.id>
```

Add `--first-frame <path>` for chained mode (N-1's last frame).

**On success:** append a new clip to `tracks[0].items`:

```json
{
  "id": "clip-<scene.id>",
  "type": "video",
  "src": "<returned path>",
  "start": <cumulative sum of prior durations>,
  "end": <start + scene.duration>,
  "inPoint": 0,
  "outPoint": <scene.duration>,
  "generation": {
    "sceneId": "<scene.id>",
    "provider": "kling",
    "model": "<chosen model>",
    "prompt": "<combined>",
    "refImages": ["<ref_id>", ...],
    "duration": <scene.duration>,
    "attempts": []
  }
}
```

**These clips have no editing proxy, and neither does the one `kling_generate` writes for you.** `lib/ai_video.py:235-253` builds the item with `id`/`type`/`src`/`start`/`end`/`inPoint`/`outPoint`/`generation`/`sourceDuration` and no `proxySrc`, so an `ai_video` project fails the WebCodecs eligibility gate on every clip (`montaj_assets/editor/src/engine/eligibility.ts:69` refuses the whole project when any track-0 video item lacks the field). Do not invent a value for it and do not compute the path yourself — Step E.2 below backfills them all in one call, after every scene has landed.

The prompt stored on `generation.prompt` is the **caller's composed string** (styleAnchor + scene prose in natural language) — NOT the wire string the connector produced after prepending its ref clause. The connector derives the ref clause deterministically from `refImages`, so regen can re-run the same caller prompt and reproduce the same wire string. Phase 7's regenerate flows (full-scene and subcut) pre-fill the prompt field from this.

**Post-download normalization:** After a clip is saved, `save_clip_to_project` automatically probes it and normalizes to the project's resolution and fps (from `project.settings`) if they don't match. For example, Kling outputs 1280x720 H.264 clips, which are upscaled to the project resolution (typically 1920x1080). The normalized file is written alongside the original (`*_normalized.mp4`) and `clip.src` is updated to point to it. If normalization fails, the original clip is kept as-is.

**Write `project.json` back IMMEDIATELY after each scene completes** — do not batch writes. The UI watches for changes via SSE and flips scene chips from "pending" → "done" in real time. If you wait until all scenes finish to write, the user sees no progress for minutes. Each `kling_generate` call returns → append the clip to `tracks[0].items` → save `project.json` → move to the next result. When running scenes in parallel, save after EACH result lands (not after all parallel calls complete).

**On failure:** record `storyboard.scenes[i].lastError = {ts: <ISO8601>, message: <error>}` and write `project.json` back immediately. Do NOT append to `tracks[0].items`. Continue to the next scene. The UI updates in real-time via SSE.

**On retry after failures:** Before retrying failed scenes, clear `lastError` on each scene you're about to retry — set `storyboard.scenes[i].lastError = undefined` and write `project.json`. This resets the UI's red "failed" chips back to "pending" so the user sees live progress. Then proceed with generation as normal. On success, the clip write naturally clears the failed state in the UI.

#### Batched (multi-shot customize)

Build `multi_prompt` JSON from the batch:

```json
[
  {"index": 1, "prompt": "<combined for scene_A>", "duration": "3"},
  {"index": 2, "prompt": "<combined for scene_B>", "duration": "4"},
  ...
]
```

Each `prompt` is the combined styleAnchor + scene prose + inline `<<<image_N>>>` tokens (SAME composition as single-shot, but respect the 512-char per-shot cap). Call:

```
kling_generate \
  --multi-shot \
  --shot-type customize \
  --multi-prompt '<json>' \
  --aspect-ratio <storyboard.aspectRatio> \
  --ref-image <path> [--ref-image <path> ...] \
  --out <path> \
  --external-task-id batch-<first_id>-<last_id>
```

Refs passed apply to any shot in the batch. Cap still 7 total.

**On success:** append ONE clip to `tracks[0].items` representing the whole batch:

```json
{
  "id": "batch-<first_scene_id>-<last_scene_id>",
  "type": "video",
  "src": "<returned path>",
  "start": <cumulative>,
  "end": <start + total_batch_duration>,
  "inPoint": 0,
  "outPoint": <total_batch_duration>,
  "generation": {
    "provider": "kling",
    "model": "<chosen model>",
    "multiShot": true,
    "shotType": "customize",
    "refImages": ["<ref_id>", ...],
    "attempts": [],
    "batchShots": [
      {"sceneId": "scene_A", "index": 1, "prompt": "<combined_A>", "start": 0.0, "end": 3.0, "duration": 3},
      {"sceneId": "scene_B", "index": 2, "prompt": "<combined_B>", "start": 3.0, "end": 7.0, "duration": 4}
    ]
  }
}
```

Same proxy caveat as the single-shot clip above: a batched clip carries no `proxySrc` either. Step E.2 covers both.

`batchShots[i].start` / `end` are **relative to the batch clip**, not the project timeline. The UI uses these for per-scene progress.

**On failure (batch-level):** the whole batch is lost. Record `storyboard.scenes[i].lastError = {ts, message, batchId}` on EVERY scene in the batch. Do NOT append to `tracks[0].items`. The user can re-click Approve (Step A skips scenes with existing clips — retry is automatic for batches with no clip) or edit individual prompts and retry.

### Step E — Audio generation, then wrap up

**Important: generate audio BEFORE setting status to `draft`.** Setting `draft` routes the user to ReviewView — audio tracks must already be on the project by then.

After all scenes have clips on `tracks[0].items`, process the audio intake fields first (Step E.1), then set status (Step E.2).

#### Step E.1 — Audio generation and assembly

Process the audio intake fields. Compute total video duration:

```python
total_duration = sum(scene['duration'] for scene in storyboard['scenes'])
```

**Re-run cleanup.** Phase 6 may run multiple times. Before appending any generated tracks, remove prior Phase-6-generated audio:

```python
project['audio']['tracks'] = [
    t for t in project['audio'].get('tracks', [])
    if not (t['id'] == 'voiceover' or t['id'].startswith('music-') or t['id'] == 'music')
]
```

#### Music

If `storyboard.music` is set:

**Upload mode** (`storyboard.music.mode === 'upload'`):
- Probe the file: `run_step('probe', { 'input': storyboard.music.path })` → get `duration`.
- Append one `AudioTrack`:
  ```json
  {
    "id": "music",
    "src": "<storyboard.music.path>",
    "start": 0,
    "end": min(duration, total_duration),
    "sourceDuration": duration,
    "volume": 0.3,
    "label": "music (uploaded)",
    "ducking": { "enabled": true }
  }
  ```

**Describe mode** (`storyboard.music.mode === 'describe'`):
- Call `run_step('generate_music', { prompt: storyboard.music.prompt, out: '<project_dir>/assets/music.wav' })`.
- Lyria returns a fixed-length track (~68 s for lyria-3.5). If `total_duration > duration`, tile the track by creating multiple `AudioTrack` entries pointing to the same file at sequential start offsets:
  ```python
  start = 0
  while start < total_duration:
      seg_end = min(start + dur, total_duration)
      append AudioTrack with id=f"music-{start}", src=result.path,
        start=start, end=seg_end, inPoint=0, outPoint=seg_end-start,
        sourceDuration=dur, volume=0.3, label=f"music (generated, loop at {start}s)",
        ducking={ enabled: true }
      start += dur
  ```

**Ducking config:** `{ enabled: true }` is sufficient. The render pipeline's `mix-audio.js` applies defaults for depth (−12 dB), attack (0.3s), and release (0.5s) when those fields are absent.

#### Voiceover

If `storyboard.voiceover` is set:

**Step 1 — decide script vs. brief.** Inspect `storyboard.voiceover.prompt`:
- If the text reads like literal spoken lines (first-person narrative, quoted dialogue) → use verbatim as TTS input.
- If the text reads like a direction ("narrate like a documentary") → expand into a full script via LLM call, sized to ~`total_duration * 150 / 60` words (~150 wpm narration pace).
- Ambiguous cases: prefer verbatim (trust the user's text).

**Concrete examples:**

| User prompt | Interpretation | Action |
|---|---|---|
| `"Welcome to our farm, where every morning begins with..."` | First-person narrative, reads as spoken lines | **verbatim** — feed directly to TTS |
| `"narrate like David Attenborough describing a quiet morning"` | Clear direction, no content | **expand** — LLM generates a script in that voice |
| `"the dog looks up at the sky"` | Third-person description, could go either way | **verbatim** (prefer trusting the user) |
| `"make it sound urgent and dramatic, mention the storm"` | Direction + content hint | **expand** — LLM writes an urgent script about a storm |

**Step 2 — call the step (with Kling→Gemini fallback):**

**Voice selection.** `TTS_VOICES` in `connectors/kling.py` is currently empty (placeholder IDs). Pass the raw voice string directly — the connector falls back to using it as-is via `TTS_VOICES.get(voice, voice)`. For Gemini, use documented voice names.

| Inferred tone | Kling `--voice` | Gemini `--voice` |
|---|---|---|
| Neutral / documentary / instructional | `"female_warm"` (raw string; Kling resolves or uses default) | `"Kore"` |
| Energetic / commercial / dramatic | `"female_warm"` | `"Puck"` |
| Dark / serious | `"male_calm"` | `"Charon"` |

Always record the chosen voice on the resulting track's label (e.g. `"voiceover (kling:female_warm)"` or `"voiceover (gemini:Kore)"`).

```python
out = f"{project_dir}/assets/voiceover.wav"

# Primary: Kling TTS. Fall back to Gemini if Kling errors.
try:
    result = run_step('generate_voiceover', {
        'text':   script,
        'voice':  'female_warm',       # raw string — Kling connector passes through
        'out':    out,
        'vendor': 'kling',
    })
    voice_label = 'kling:female_warm'
except Exception as e:
    # Kling TTS failed (likely due to placeholder voice IDs) — retry with Gemini.
    agent_log(f"Kling TTS failed ({e}); retrying with Gemini TTS")
    result = run_step('generate_voiceover', {
        'text':   script,
        'voice':  'Kore',              # Gemini documented voice name
        'out':    out,
        'vendor': 'gemini',
    })
    voice_label = 'gemini:Kore'
```

**Step 3 — append as track:**
```json
{
  "id": "voiceover",
  "src": "<result.path>",
  "start": 0,
  "end": min(result.duration_seconds, total_duration),
  "sourceDuration": result.duration_seconds,
  "volume": 1.0,
  "label": "voiceover (<voice_label>)",
  "ducking": { "enabled": false }
}
```

**Duration mismatch handling:**
- VO duration > total_duration: clamp `end` to `total_duration` (truncates tail). Warn the user.
- VO duration < total_duration: VO plays and stops; silence fills the remainder (with music still playing if present). No warning needed.

**Error handling:** If `generate_music` or `generate_voiceover` fails, skip the failed track — do not abort the whole project. Surface the error to the user. Continue with the other track if available.

Write `project.json` after appending the audio tracks.

#### Step E.2 — Set status

- **When every `storyboard.scenes[i]` has a matching clip** (by sceneId OR batchShots sceneId) AND audio generation is complete (or skipped if no intake fields): **first backfill the editing proxies, then** set `project.status = "draft"`. The UI's EditorPage routing carries the user to ReviewView.

  ```bash
  curl -s -X POST http://localhost:3000/api/projects/{id}/proxies
  ```

  Every clip on `tracks[0]` was written without a `proxySrc` — neither `kling_generate` nor the hand-written batched and subcut shapes add one. Without them the WebCodecs playback engine refuses the project (`montaj_assets/editor/src/engine/eligibility.ts:69`), preview falls back to decoding each generated master on every seek, and the header shows a chip telling the operator their clips have no previews. This one call covers the whole project: it computes each proxy path itself, de-duplicates by target, encodes in the background, and writes `proxySrc` back onto each item over SSE. `202` means encodes were queued; `200` means everything was already fresh. It does not block — flipping to `draft` immediately afterwards is correct, and the timeline fills in as the encodes land.

  Do this on Phase 7 regen too, after a `mode: "full"` or `mode: "subcut"` entry has rewritten `tracks[0].items` — the new pieces are new files and need their own proxies.
- **If some scenes failed:** leave status at `storyboard_ready`. The user sees partial progress (some cards "done," some showing error). They may re-click Approve (idempotency handles retry) or ask in chat for tweaks.

### Step F — evaluate generated clips (optional but recommended)

After all scenes have clips, run the eval loop:

    for each scene:
      eval_scene --project-id {id} --scene-id {scene.id} --max-retries 2

The step evaluates each clip against a 5-dimension quality rubric (character match, physics, anatomy, action, standalone) using Gemini. If a clip fails, the step regenerates it via Kling (non-deterministic re-roll with the same composed prompt) and re-evaluates, up to `max-retries` times. Previous attempts are preserved as versioned files (`scene-1-v2.mp4`, etc.) and recorded in `generation.attempts[]` with their eval verdicts.

Results are stored on `tracks[0].items[i].generation.eval` — `{pass, scores, attempt}`.

**When to skip:** If the user says they're happy with the clips, want to iterate manually, or are cost-sensitive (each eval = 1 Gemini call + potentially N Kling calls per scene).

**Note:** The eval loop does NOT revise prompts — it re-rolls generation with the same prompt, relying on Kling's non-determinism to produce a better draw. Prompt revision based on Gemini feedback is a future enhancement.

---

## Phase 7 — Draft phase (status: `draft`)

Treat the project like any other. Respond to timeline-editing requests.

Don't touch the `generation` block on `tracks[0].items` clips unless explicitly asked ("regenerate scene 3 with a different prompt"). That block is a frozen snapshot of what produced the clip.

### Processing the regenQueue

`project.regenQueue[]` is the authoritative queue for per-clip regeneration requests. The UI (inspect modal for full-scene, subcut range-picker tool for windowed regens) and the `montaj regen` / `montaj regen subcut` CLI commands all write entries to this queue. When the user triggers you — typically with *"Please process project.regenQueue[] per the ai-video-generate skill Phase 7 contract."* — drain it.

**Verification guard (same spirit as Phase 6's approval guard):**
1. `project.projectType === "ai_video"`.
2. `project.regenQueue` is a non-empty array.
3. Each entry's `clipId` matches a real item in `tracks[0].items`.

Bad entries → tell the user, do NOT drop silently.

**For each entry, in order:**

1. **Locate the clip.** `clip = tracks[0].items.find(c => c.id === entry.clipId)`. If missing → set `entry.lastError = {ts, message: "clip not found"}`, continue to next entry.

2. **Extract continuity frames if needed (subcut + toggle on):**
   - `entry.useFirstFrame === true` → `snapshot --input <clip.src> --at <entry.subrange.start> --out <frame_first.jpg>`.
   - `entry.useLastFrame === true` → `snapshot --input <clip.src> --at <entry.subrange.end> --out <frame_last.jpg>`.

3. **Compose the full prompt** from `entry.prompt` + `entry.refImages`, then **run the `kling_generate` step**.

   The connector is a pure pass-through — you must place `<<<image_N>>>` tokens and the ref clause yourself, same as Phase 6 Step C. Compose the prompt:
   - Start with `storyboard.styleAnchor` (if present) as prefix.
   - Append `entry.prompt`. At each character/object label that matches a ref's `imageRefs[i].label`, insert the positional `<<<image_N>>>` token (N = 1-indexed, matching `--ref-image` order).
   - Prepend the ref clause: `"Use the character/style from <<<image_1>>>, <<<image_2>>>. "`.

   Store the **pre-composition** natural-language prompt (without tokens/clause) on `generation.prompt` when patching the clip.

   Run the `kling_generate` step with the composed prompt:
   - `--prompt "<composed prompt with tokens and ref clause>"`
   - `--duration <entry.duration>`
   - `--aspect-ratio <project.storyboard.aspectRatio>`
   - `--model <entry.model>` (falls back to `kling-v3-omni` if unset)
   - `--ref-image <resolved_path>` per ID in `entry.refImages` (resolve via `project.storyboard.imageRefs[i].refImages[0]`)
   - `--first-frame <frame_first.jpg>` / `--last-frame <frame_last.jpg>` if step 2 produced them
   - `--out <workspace_path>`
   - `--external-task-id <entry.id>`

4. **Patch `tracks[0].items` based on mode:**

   **`mode: "full"`** — replace in place:
   - Push `{ts, prompt: <clip.generation.prompt>, src: <clip.src>}` onto `clip.generation.attempts[]`.
   - `clip.src = <new path>`, `clip.inPoint = 0`, `clip.outPoint = entry.duration`.
   - `clip.end = clip.start + entry.duration` → ripple subsequent items.
   - Update `clip.generation.prompt = entry.prompt`, `.duration = entry.duration`, `.refImages = entry.refImages`, `.model = entry.model`.

   **`mode: "subcut"`** — split and insert:
   - **Left piece** (if non-zero duration): `inPoint = clip.inPoint`, `outPoint = entry.subrange.start`; timeline span `start = clip.start`, `end = clip.start + (subrange.start - clip.inPoint)`. Inherits `clip.generation` unchanged.
   - **Middle piece** (new): `src = <new path>`, `inPoint = 0`, `outPoint = entry.duration`; timeline span starts where left ends. Fresh `generation`: `{sceneId: clip.generation.sceneId, provider: "kling", model: entry.model, prompt: entry.prompt, refImages: entry.refImages, duration: entry.duration, attempts: []}`.
   - **Right piece** (if non-zero duration): `inPoint = entry.subrange.end`, `outPoint = clip.outPoint`; timeline span starts where middle ends. Inherits `clip.generation` unchanged.
   - Replace the single `tracks[0].items` entry with the non-degenerate pieces in order. Ripple subsequent clips.
   - Degenerate case: subrange covers the whole clip → no left/right → equivalent to full-scene regen. No special-case logic needed.

5. **Remove the processed entry** from `regenQueue`. Write `project.json`.

6. Loop to the next entry.

**On failure (any entry):** do NOT remove it. Set `entry.lastError = {ts: <ISO8601>, message: <kling error>}` and continue to the next entry. User can re-trigger after investigating.

**Regen contract — what NOT to do:**
- Don't process `regenQueue` entries before the user triggers you. The queue being non-empty doesn't mean "go" — wait for the chat message, same pattern as `storyboard.approval` in Phase 6.
- Don't process entries in parallel if they target overlapping clips. Sequential avoids races.
- Don't mutate `storyboard.scenes[]` during regen. Editorial plan stays stable; regen operates on `tracks[0].items` only.
- Don't drop bad entries silently. Record `lastError` so the user can fix and retry.
- Don't pass `entry.prompt` raw as `--prompt` — compose it first (tokens + ref clause + styleAnchor). The connector does NOT add tokens; you must.
- Don't skip the snapshot step when `useFirstFrame` / `useLastFrame` is set. Those toggles are the user's explicit continuity request; honor them.

---

## Field path reference

One table of every field you write, grouped by phase.

| Field | When | What |
|-------|------|------|
| `project.storyboard.scenes[i].lastError` | phase 6 | Optional per-scene error record. Clear on eventual success. For batched failures, stamp on every scene in the failed batch. |
| `project.tracks[0].items` | phase 6 | **Append** real clips only. Never stubs, never empty-src items. |
| `project.tracks[0].items[i].generation.*` (single-shot) | phase 6 | Frozen snapshot: `{sceneId, provider, model, prompt, refImages, duration, attempts}`. |
| `project.tracks[0].items[i].generation.multiShot / shotType / batchShots` | phase 6, batched only | Batched clip metadata; `batchShots[]` carries per-scene mapping inside the concatenated output. |
| `project.tracks[0].items[i].generation.attempts` | post-draft regen (Phase 7 regenQueue drain) | Append previous `{ts, prompt, src}` before overwriting on a `mode: "full"` regen. For `mode: "subcut"`, the middle piece is a fresh clip with empty `attempts[]`; the left/right pieces inherit the original clip's attempts unchanged. |
| `project.regenQueue` | Phase 7 | UI and CLI append entries. Agent drains on trigger — remove on success, set `lastError` on failure. |
| `project.storyboard.music` | intake | `{ mode: 'upload', path }` or `{ mode: 'describe', prompt }`. Project-wide music brief. |
| `project.storyboard.voiceover` | intake | `{ prompt }`. Project-wide voiceover script or brief. |
| `project.audio.tracks` (music/voiceover entries) | phase 6 | Appended after all scene clips are on `tracks[0].items`. Music at volume 0.3 with ducking enabled; voiceover at volume 1.0. Stripped and re-created on Phase 6 re-runs. |
| `project.status` → `draft` | end of Phase 6 | Only when every `storyboard.scenes[i]` has a matching clip (single-shot sceneId OR batchShots sceneId). |

---

## What NOT to do

- **Don't generate scenes sequentially in independent mode.** Fire all `kling_generate` calls in parallel (up to 4 concurrent). Sequential one-at-a-time dispatch wastes minutes of wall-clock time — Kling handles concurrent requests fine.
- **Don't use multi-shot mode with `--first-frame` / `--last-frame`.** Kling doesn't support frame control in multi-shot. If you need frame continuity, use chained mode.
- **Don't exceed 512 chars per shot in multi-shot mode.** Single-shot's 2500-char budget does NOT apply per-shot. Fall back to independent mode for any scene that needs more prose.
- **Don't use `sceneId` on the outer `generation` block of a batched clip.** Use `batchShots[].sceneId`. The idempotency check in Step A branches on whether `batchShots` exists.
- **Don't set non-integer `scenes[i].duration`.** Kling's per-scene enum is integers (3–15 single-shot, 1–N multi-shot). Floats get silently clamped; apparent length drifts from your plan.
- **Don't run `kling_generate` before `storyboard.approval` is set.** Premature generation wastes credits.
- **Don't overwrite `storyboard.approval`.** The UI owns that field.
- **Don't mutate `tracks[0].items[i].generation` to "edit the scene".** That block is a frozen snapshot. To change editorial intent, edit `storyboard.scenes[i]`. To actually regenerate with new settings, queue a `regenQueue` entry (via UI, CLI, or by writing one yourself in chat) and drain per Phase 7.
- **Don't regenerate scenes that already have a clip** when the user re-approves. The skip-if-clip-exists check in Phase 6 Step A is mandatory — partial-approval retries depend on it.
- **Don't put scenes on `tracks[0].items` before they've been generated.** `tracks[0].items` holds real clips only — no stubs, no empty-src items.
- **Don't set `project.status` to `"draft"` until every `storyboard.scenes[i]` has a matching clip in `tracks[0].items`.**
- **Don't chain scenes via `--first-frame` by default.** Hard cuts are the standard.
- **Don't include `## Dialogue` in scene prompts when `storyboard.voiceover` is set.** Kling-generated dialogue competes with TTS voiceover — omit it so scenes only produce ambient SFX.
- **Don't skip the Phase 6 re-run cleanup for audio tracks.** Always strip prior `music`, `music-*`, and `voiceover` tracks before appending new ones — prevents stale accumulation on re-approval.
- **Don't invent fields outside the schema.** If you need to store something, ask the user.
- **Don't touch `project.json` outside ai_video's documented paths** (e.g., don't add fields to `settings`).