venice-video-model-routing · git:20260920.fd110a8 · 2026-09-20 · sha256 23ddc57f4c9520d1
venice-video-model-routing git:20260920.fd110a8A
Immutable. This exact content never changes and is served at /api/v1/blob/23ddc57f4c9520d1.
---
name: venice-video-model-routing
description: This skill should be used when generating images or videos via Venice AI, selecting models, attaching reference images, choosing frame sources, or adapting prompts for character consistency. Includes executable Python scripts for image generation, video generation, image editing, and upscaling. Covers R2V-first model routing (characters present → R2V, no characters → action/atmosphere), elements vs reference_image_urls, frame chaining vs panel start, two-pass image pipeline routing, and prompt adaptation per model. Applicable to any project generating multi-shot video content via the Venice AI API.
metadata:
requires:
bins: ["python3"]
env: ["VENICE_API_KEY"]
---
# Venice Video Model Routing For Character Consistency
Route Venice AI models, reference image strategies, frame sources, and prompt formats to maximize character consistency across image panels and video clips. Includes bundled Python scripts for executing image generation, video generation, image editing, and upscaling via the Venice AI API.
## Prerequisites and Setup
### Requirements
- **Python 3.10+** (`brew install python` or system Python)
- **Venice API key** (free tier available at [venice.ai](https://venice.ai))
### Get Your API Key
1. Create an account at [venice.ai](https://venice.ai)
2. Go to [venice.ai/settings/api](https://venice.ai/settings/api)
3. Click "Create API Key"
4. Copy the key (starts with `vn_...`)
### Configure the Key
**Option A: Environment variable**
```bash
export VENICE_API_KEY="vn_your_key_here"
```
**Option B: `.env` file** in your project root:
```
VENICE_API_KEY=vn_your_key_here
```
### Verify Setup
```bash
python3 {baseDir}/scripts/venice-image.py --list-models
python3 {baseDir}/scripts/venice-video.py --list-models
```
## Generation Scripts
Four Python scripts handle Venice API execution. Use these to generate media, then apply the routing decisions in Sections 1-7 to choose models and parameters.
### Image Generation
```bash
# Generate a storyboard panel
python3 {baseDir}/scripts/venice-image.py \
--prompt "STYLE: Korean manhwa... [scene content] ...STYLE REMINDER: manhwa, cel-shaded" \
--model nano-banana-pro \
--resolution 1K --aspect-ratio 9:16 \
--cfg-scale 10 --seed 88442211 \
--steps 30 --hide-watermark
# Generate character references (square)
python3 {baseDir}/scripts/venice-image.py \
--prompt "Front-facing reference sheet..." \
--model nano-banana-pro \
--resolution 1K --aspect-ratio 1:1
# List available image models
python3 {baseDir}/scripts/venice-image.py --list-models
# Generate multiple variants
python3 {baseDir}/scripts/venice-image.py --prompt "..." --count 4
```
Key flags: `--model`, `--prompt`, `--resolution` (1K/2K), `--aspect-ratio` (9:16, 16:9, 1:1), `--cfg-scale` (0-20, default 7.5), `--seed`, `--steps`, `--negative-prompt`, `--style-preset`, `--hide-watermark`, `--count`, `--out-dir`
### Video Generation
```bash
# Generate video from panel (basic -- text-only model)
python3 {baseDir}/scripts/venice-video.py \
--image shot-001.png \
--prompt "A slow dolly shot pushes forward..." \
--model kling-v3-pro-image-to-video \
--duration 5s \
--audio
# Get price quote before generating
python3 {baseDir}/scripts/venice-video.py --quote \
--model kling-o3-standard-reference-to-video --duration 8s --aspect-ratio 9:16
# List available video models with durations
python3 {baseDir}/scripts/venice-video.py --list-models
```
Key flags: `--image` (source panel), `--prompt`, `--model`, `--duration` (3s-16s, model-dependent), `--resolution` (480p/720p/1080p), `--aspect-ratio`, `--audio`/`--no-audio`, `--quote`, `--timeout`, `--poll-interval`, `--out-dir`
**Limitation:** The bundled video script handles basic image-to-video generation. For advanced routing features (`elements`, `reference_image_urls`, `image_urls`, `end_image_url`), construct the API call directly using the Venice REST API. See the capability matrix in Section 3 for which models support which parameters.
### Image Editing (Multi-Edit / Refinement)
```bash
# Fix character appearance in a panel
python3 {baseDir}/scripts/venice-edit.py shot-001.png \
--prompt "Make the woman match the reference: tall, hourglass figure, dark hair..."
# Edit from URL
python3 {baseDir}/scripts/venice-edit.py --url "https://..." \
--prompt "change the sky to sunset"
```
Key flags: `image` (positional, local path), `--url` (remote image), `--prompt` (edit instruction), `--output`, `--out-dir`
### Image Upscaling
```bash
# 2x upscale
python3 {baseDir}/scripts/venice-upscale.py shot-001.png --scale 2
# 4x upscale with AI enhancement
python3 {baseDir}/scripts/venice-upscale.py shot-001.png --scale 4 --enhance
# Upscale with detail sharpening
python3 {baseDir}/scripts/venice-upscale.py shot-001.png --enhance --enhance-prompt "sharpen details"
```
Key flags: `image` (positional), `--url`, `--scale` (1-4), `--enhance`, `--enhance-prompt`, `--enhance-creativity` (0.0-1.0), `--replication` (0.0-1.0), `--output`, `--out-dir`
### Pricing Overview
| Feature | Cost |
|---------|------|
| Image generation | ~$0.01-0.03 per image |
| Image upscale | ~$0.02-0.04 |
| Image edit | ~$0.04 |
| Video (WAN) | ~$0.10-0.50 depending on duration |
| Video (Kling) | ~$0.20-1.00 depending on duration |
| Video (Sora) | ~$0.50-2.00 depending on duration |
Use `--quote` with the video script to check pricing before generation.
---
## 1. Video Model Routing: Seedance R2V by Default
**Core principle: Seedance R2V by default, Kling O3 R2V fallback for 3+ character scenes, Seedance i2v for establishing shots.**
Almost all shots should use a reference-to-video model for identity anchoring. Seedance 2.0 R2V (#1 ranked on Artificial Analysis) is the default, using flat `reference_image_urls` with `@Image` prompt tags. For shots with 3+ characters, the system falls back to Kling O3 R2V which provides structured `elements` for better per-character identity separation. Empty establishing/mood shots use Seedance i2v for superior cinematic quality and physics.
| Role | Default Model | When Used | Capabilities |
|------|--------------|-----------|--------------|
| **Character shots (1-2 chars)** | `seedance-2-0-reference-to-video` | Default R2V — flat `reference_image_urls` with `@Image` tags | Up to 4 ref images, 4-15s, native stereo audio, `aspect_ratio` |
| **Character shots (3+ chars)** | `kling-o3-standard-reference-to-video` | Auto-fallback — structured `elements` for multi-character identity | `elements`, `reference_image_urls`, `scene_image_urls`; 3-15s |
| **Establishing / mood / action** | `seedance-2-0-image-to-video` | No characters — epic cinematic quality, physics-aware | 4-15s, `resolution: '720p'`, `aspect_ratio`, native audio |
Recommended constants (harness default is Seedance 2.5 R2V for ALL lanes since 2026-08-07; see AGENTS.md rule 51):
```
ACTION_MODEL = 'seedance-2-5-reference-to-video'
ATMOSPHERE_MODEL = 'seedance-2-5-reference-to-video'
CHARACTER_CONSISTENCY_MODEL = 'seedance-2-5-reference-to-video'
MONTAGE_MODEL = 'seedance-2-5-reference-to-video' (default scene lane, up to 30s; resolveMontageModel())
MULTISHOT_MODEL = 'seedance-2-5-reference-to-video' (montage-off lane, up to 15s; resolveMultiShotModel())
KLING_R2V_MODEL = 'kling-o3-standard-reference-to-video' (rare fallback — 2.5 carries up to 30 refs)
# Seedance 2.5: every integer 4-30s, 480p/720p (harness pins 720p), up to 30 refs,
# @Image tags, audio_url + reference_audio_urls + reference_video_urls.
# Prior default 'seedance-2-0-enhanced-reference-to-video' (1080p-capable) stays
# selectable via videoDefaults.
# Legacy multi-shot: 'kling-o3-pro-image-to-video' — explicit videoDefaults.multiShotModel
# override only. It has NO elements and NO reference_image_urls, so it drops all identity anchoring.
# Single image default for ALL panels (Venice removed the Seedance face restriction, 2026-07)
IMAGE_GENERATION = 'nano-banana-2' # character + faceless panels alike
MULTI_EDIT = 'nano-banana-2-edit' # character fixes + style-match
```
### Seedance Face Rule — REMOVED (2026-07)
Seedance 2.0 used to reject face-bearing input images that weren't produced by `seedream-v5-lite` / `seedream-v5-lite-edit`. **Venice removed that restriction** — Seedance now accepts face-bearing images from **any** image family. There is no longer a seedream requirement: use the global default (`nano-banana-2`) for character portraits, panels, and references and send them straight to Seedance.
The harness still writes a provenance sidecar (`<image>.provenance.json`) recording `{ generationModel, editModels[], hasFace }` as metadata, but nothing gates on it. `ensureSeedanceCompatibility()` (`src/venice/seedance-preflight.ts`) is now a no-op that always proceeds, and `series.videoDefaults.seedanceCompatibility` is no longer auto-set. (The separate Seedance face *consent* attestation, 409 `needs_consent`, is still handled at queue time.)
## 2. Video Model Routing Decision
Three-path routing based on character count:
| Priority | Condition | Result | Reason |
|----------|-----------|--------|--------|
| 1 | Establishing/mood shot with no characters | Seedance i2v | No characters to anchor — best cinematic quality + physics |
| 2 | 1-2 characters | Seedance R2V | Identity anchoring via flat `reference_image_urls` — #1 ranked quality |
| 3 | 3+ characters | Kling O3 R2V | Structured `elements` handle more identities (up to 7 elements vs 4 flat refs) |
When Seedance R2V is selected, `reference_image_urls` and `@Image` prompt tags are auto-enabled. When Kling O3 R2V is selected (3+ char fallback), `elements` and `@Element` tags are auto-enabled.
## 3. Reference Image Attachment Matrix
Three reference mechanisms exist for video generation, each supported by different models:
### `elements` (Structured Per-Character)
- **Supported by:** `kling-o3-standard-reference-to-video`, `kling-o3-pro-reference-to-video`
- **Structure:** Array of up to 4 element definitions, each with `frontal_image_url` and up to 3 `reference_image_urls` (side, 45°, back angles)
- **Optional:** `video_url` for motion/voice reference
- **Prompt integration:** Replace character names with `@Element1`, `@Element2` tokens in both description and dialogue
- **Typical reference images:**
- `frontalImageUrl` = character front-facing reference
- `referenceImageUrls` = three-quarter, profile, and back angle references (up to 3)
- **When to use:** Consistency model selected AND model supports elements
### `reference_image_urls` (Flat General)
- **Supported by:** `seedance-2-0-reference-to-video` (+ `-enhanced-` / `-fast-` variants), `happyhorse-1-1-reference-to-video`, `kling-o3-standard-reference-to-video`, `kling-o3-pro-reference-to-video`, `vidu-q3-image-to-video`
- **Structure:** Flat array — **up to 9** on the Seedance 2.0 R2V family and HappyHorse 1.1 R2V (`getMaxReferenceImages()` in `src/series/types.ts`); legacy 4 elsewhere
- **Prompt integration:** For Seedance/Grok Imagine R2V, use `@Image1`, `@Image2` tags. For Kling R2V, standard character names or `@Element` tokens (when elements are also active). For Vidu, standard character names.
- **Slot order (reference-first, 2026-07-30, `src/mini-drama/reference-slots.ts`):** one primary angle per character → storyboard blocking plate (protected) → location angles (wide/medium/detail) → second character angles. Overflow drops second character angles first, then trailing location angles.
- **When to use:** Always on the Seedance Enhanced default (pure reference mode — no `image_url`). For Seedance R2V, this is the primary identity mechanism (no elements). For Kling R2V, used alongside `elements` or as a standalone fallback.
### `image_urls` (Scene/Environment Anchoring)
- **Supported by:** `kling-o3-standard-reference-to-video`, `kling-o3-pro-reference-to-video`
- **Structure:** Array of up to 4 scene/environment reference images
- **Prompt integration:** Referenced as `@Image1`, `@Image2` in prompt text (indices offset when `@Image` tags are also used for character refs)
- **When to use:** Environment/style reference paths are explicitly provided AND model supports scene images
- **Note:** Seedance R2V does NOT support separate scene images. Scene context is handled through prompting only.
### Capability Matrix
| Parameter | Seedance 2.0 R2V | Seedance 2.0 i2v | Kling O3 R2V | Kling V3 Pro | Vidu Q3 | Sora 2 | Wan 2.6 | LTX 2.0 | Veo 3.1 | Longcat | PixVerse |
|-----------|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|
| `elements` | No | No | Yes | **NO** | No | No | No | No | No | No | No |
| `reference_image_urls` | **Yes** | No | Yes | **NO** | Yes | No | No | No | No | No | No |
| `scene_image_urls` | No | No | Yes | No | No | No | No | No | No | No | No |
| `end_image_url` | No | No | Yes | Yes | No | No | No | No | No | No | Transition |
| `audio_url` | No | No | No | No | No | No | Yes | No | No | No | No |
| `@Image` prompt tags | **Yes** | No | No | No | No | No | No | No | No | No | No |
| `aspect_ratio` | Yes | Yes | Yes | No | - | Yes | T2V only | Yes | T2V only | T2V only | Yes |
| `resolution` | Yes | Yes | No | No | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
| Max duration | **15s** | **15s** | 15s | 15s | 16s | 12s | 15s | 20s | 8s | **30s** | 8s |
| Audio | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | No | Yes |
| Native lip-sync | **8+ langs** | **8+ langs** | No | No | No | No | No | No | No | No | No |
Always gate reference attachments through the capability matrix. Sending unsupported params to models that reject them returns 400 errors. The model registry in `src/venice/models.ts` has full typed specs for every model.
## 4. Frame Source Strategy (Identity vs Continuity)
When generating sequential video clips, two decisions govern visual continuity:
### Start Frame (`image_url`)
**Use the panel/storyboard image** (prioritizes character identity -- the panel was refined against character references):
- First shot in the sequence (no previous video)
- `continuityPriority === 'identity'` on the shot
- Scene boundary (establishing shot, no character overlap, dramatic composition change)
- New character entering who was not in the previous shot
- Identity-sensitive shot type (`close-up`, `reaction`) unless continuity is explicitly prioritized
- Previous transition is `CUT` or `SMASH CUT`
**Use the previous video's last frame** (prioritizes visual flow -- smooth transition):
- Same characters continue across shots AND
- Previous shot's transition is in the chain set: `DISSOLVE`, `MATCH CUT`, `MORPH`, `WIPE`, `CROSSFADE`, `FADE`
### End Frame (`end_image_url`, Kling only)
**Target next panel** (Kling animates toward this composition):
- Current shot's transition is: `DISSOLVE`, `MATCH CUT`, `MORPH`, `WIPE`, or `CROSSFADE`
- AND next shot exists, has no new characters, and is not a title/insert
**Natural ending** (video ends wherever the animation takes it):
- Transition is `CUT`, `SMASH CUT`, or `FADE`
- OR next shot has new characters (end_image would show wrong appearance)
- OR next shot is a title card or insert
### Transition Quick Reference
```
Start frame chaining: DISSOLVE, MATCH CUT, MORPH, WIPE, CROSSFADE, FADE
End frame targeting: DISSOLVE, MATCH CUT, MORPH, WIPE, CROSSFADE (no FADE)
No chaining: CUT, SMASH CUT
```
## 5. Two-Pass Image Pipeline Routing
Generate consistent storyboard panels using two Venice models in sequence:
### Pass 1: Generate (`nano-banana-pro`)
- **Script:** `python3 {baseDir}/scripts/venice-image.py --model nano-banana-pro ...`
- **Reference images:** None accepted (400 error if `image_references`/`image_1` are sent)
- **Consistency via:** Exhaustive text descriptions (full character + aesthetic bookended in prompt) + fixed seed
- **Recommended params:** `--resolution 1K --aspect-ratio 9:16 --steps 30 --cfg-scale 10 --seed <fixed project seed>`
- **Output:** 768x1376 (9:16) or 1376x768 (16:9)
### Pass 2: Refine via Multi-Edit (`nano-banana-pro-edit`)
- **Script:** `python3 {baseDir}/scripts/venice-edit.py panel.png --prompt "Match character to reference..."`
- **Default model:** `nano-banana-pro-edit` (via multi-edit endpoint)
- **Input:** Base panel + up to 2 character reference images
- **Output:** Always 1024x1024 (1:1) -- restore original aspect ratio via center-crop + scale after
**Two refinement paths based on shot content:**
| Shot Type | Images Sent | Purpose |
|-----------|-------------|---------|
| Character shot (has characters) | Panel + 1-2 character front-facing references | Align face, body, wardrobe to references |
| Non-character shot (establishing, insert, title) | Panel + style anchor (a refined character shot from same project) | Harmonize rendering style across all panels |
**Available multi-edit models:**
`nano-banana-pro-edit` (default), `nano-banana-2-edit`, `gpt-image-1-5-edit`, `grok-imagine-edit`, `qwen-edit`, `flux-2-max-edit`, `seedream-v4-edit`, `seedream-v5-lite-edit`
## 6. Prompt Adaptation Per Model
Construct prompts differently depending on the resolved model's capabilities:
### Simple-Prompt Models (MiniMax H3 Max, H3 Max Turbo)
Read this before anything else in this section: everything below assumes a model that
renders what it is told and drifts when it is not. The H3 Max pair is the opposite, and
the registry marks it with `promptStyle: 'simple'` (`modelWantsSimplePrompt(modelId)`).
These models compose their own coverage — framing, cutting, beat rhythm — from one plain
statement of intent, and the directorial stack fights the shot the model would otherwise
have chosen.
`buildVideoPrompt()` and `buildMontagePrompt()` already drop the heavy blocks for them, so
the adaptation is mostly a matter of not writing them back in by hand:
- **Dropped:** the authored `Blocking:` restatement, the location description and
`spatialAnchors` "Fixed layout (never rearrange)" line, the geography-hold /
no-mirroring lecture, and the full aesthetic string (the compact one is used instead).
- **Kept:** `@ImageN` identity declarations and role clauses on the R2V lane, the beat
description, dialogue with delivery, and the audio-exclusion suffix. Identity and look
still have to be bound; only the staging instructions go.
- **Dialogue is IMPROVISED, not scripted.** These models carry natural, continuous speech
across a whole generation and degrade when handed an exact line to recite. So in
native-dialogue mode the prompt gives the scripted line as INTENT — `[@Image1, voice,
delivery] conveys: "…"` plus a one-line "improvise naturally in character, keep the intent
and tone, don't recite word for word" note — instead of the directorial `[…]: "exact line"`
quote. `buildVideoPrompt` / `buildMontagePrompt` do this automatically via
`shouldImproviseDialogue(modelId, series)` (gated on `modelWantsSimplePrompt` +
`audioStrategy !== 'lip-sync'`). Do NOT hand-write exact quotes for these models expecting
them verbatim. The exception is **exact-lip-sync**, where the `audio_url` drives the exact
words, so the line stays verbatim. One consequence: captions must come from transcribing
the rendered audio, not from `script.json` (the model won't say it word for word).
- **Don't** add camera terms, shot lists, or `Lens switch.` lines per beat. State the
sequence in a sentence or two and let the model cut it — that instinct is why this is
the montage family.
- Resolution is pinned to **768P** by default (2K is a hard 400 — the inverse of plain
MiniMax H3, which is 2K-only). **480P is the draft tier** and is only selected via an
explicit `resolution` override on `renderVideoFile`. Durations run the 5-15s ladder.
`audio` is not configurable, so the field is omitted from the body entirely.
- Turbo ships **no R2V lane**; identity and lip-sync shots cross to
`minimax-h3-max-reference-to-video`.
**Loop mode uses this family, with two modes.** `venice-video loop` renders the whole shot
script continuously and plays it as a live browser loop that hot-swaps in fresh takes:
- `--mode watch` (default) — `minimax-h3-max-turbo-text-to-video` (or `-image-to-video` off
an existing panel) at 480P, ~$0.012/s. The cheapest lane; a disposable draft that does NOT
lock identity (Turbo has no R2V lane).
- `--mode create` — the real reference-first routing on the **non-Turbo** H3 Max family at
768P (~$0.024/s): character shots on `minimax-h3-max-reference-to-video` with the full
`@Image` reference stack + voice-donor audio (identity locked, takes usable), atmosphere on
Max i2v/t2v. Shots with missing references degrade to i2v/t2v.
Both skip the storyboard/QA gates and write only under `loop/`. See `LoopEngine`
(`src/mini-drama/loop-engine.ts`) and AGENTS.md rule 58. Watch is a preview, not a production
path; create is the "keep the good takes" path.
**The loop plays AND renders concurrently — it does not pre-generate.** The browser loops the
takes that already exist while the worker keeps generating new ones, swapping each shot's newer
take in on the loop's NEXT pass (never mid-clip). It only *looks* pre-generated when (a) it is
PAUSED — budget hit or Pause clicked — and just replays what is on disk (raise `--budget` or
pass `--unbounded` to keep it going), or (b) it is in `create` mode, whose slow Max R2V renders
can't keep up with a short loop cycle, so evolution lags. `watch` (looping/Turbo) renders faster
than it plays and visibly evolves as you watch; `create` (production) is for gathering keeper
takes, not a live-evolving watch. `--max-takes` is a per-shot ring buffer, not a total/stop.
The Creator app mirrors this: `VideoModelCapabilities.wantsSimplePrompt(id:)` gates the
same trimming in `ShotPromptBuilder`, and relaxes the `produce_shots` motion/length gate
so a correctly short H3 Max prompt isn't rejected as thin.
### Camera-Path Models (MiniMax H3 Max Multi-Angle)
`minimax-h3-max-multi-angle` is a simple-prompt i2v lane with one thing no other
Venice model has: a `camera_trajectory`. You give it the start frame (`image_url`)
and a **2–12 keyframe** camera path, and it orbits the subject along it. `prompt`
is optional — the trajectory carries the shot.
- Each keyframe is `{ time (0–1, strictly increasing), azimuth° (horizontal,
signed), elevation° (vertical, −90..90), distance (>0, 1 = unchanged) }`. Total
absolute azimuth travel ≤ **32 full turns** (11520°). Build with
`buildStartEndTrajectory(start, finish)` (the literal start→finish form) or
`buildOrbitTrajectory({ azimuthTravel, ramp, startElevation, endElevation,
startDistance, endDistance, keyframes })` from `src/venice/models.js`.
- **In-shot speed ramping** = uneven angular velocity over even time. Pass
`ramp: 'ease-in' | 'ease-out' | 'ease-in-out'` (with `keyframes ≥ 3`); the
builder eases azimuth across evenly spaced `time` so the rotation accelerates
or settles without breaking the strictly-increasing-time rule.
- **Resolution:** the ONE H3 Max lane that renders **1080P** (`['1080P','768P',
'480P']`; base H3 Max caps at 768P). The auto-pin still defaults 768P for cost;
pass `resolution: '1080P'` for a finish render or `'480P'` for a cheap draft.
- i2v only — aspect follows the start image (send no `aspect_ratio`), audio is on
and NOT configurable (omit the `audio` field), 5–15s ladder, `private` +
uncensored. No t2v/R2V multi-angle lane.
- Wire via `generateVideo`/`queueVideo` (`cameraTrajectory`) or `renderVideoFile`
(`cameraTrajectory`); `validateCameraTrajectory` fails a bad path fast, before a
paid queue. Anti-pattern: sending `camera_trajectory` to any other model →
`400` "Unrecognized key(s)".
### Image-Tag R2V Models (Seedance 2.0 R2V Enhanced — default for all lanes)
- Replace character names in descriptions with `@Image1`, `@Image2` tokens via regex
- Declare each character up front: `@Image1 is Bob — wearing …` (rule 37 trait restatement)
- Dialogue speaker uses image ref: `[@Image1, voice description, delivery]: "dialogue line"`
- Attach `reference_image_urls` at the API layer in the slot-planner order (up to 9: character primaries → storyboard blocking plate → location angles → second character angles)
- Emit a role clause per non-character slot: location angles ("@Image4 is a second angle of the same location"), blocking plates ("use ONLY for composition, blocking, and spatial relationships")
- Carry the shot's authored `blocking` verbatim (`Blocking: @Image1 at the bar counter, screen left, facing right; …`) and the location's `spatialAnchors` ("Fixed layout (never rearrange): …") — placement is stated per shot, never left to the model (rule 49)
- With a blocking plate bound, the prompt also pins geometry to it: characters keep their side of the scene and position relative to the plate's landmarks — no mirroring, swapping, or rearranging; plateless location shots pin geography to the first location `@ImageN` slot instead
- **Pure reference mode:** no `image_url` start frame when the slot plan is populated — references carry all consistency
- Keep prompts concise. Use the 5-part structure: Subject, Action, Camera, Style, Constraints.
- Use physics-aware language: describe forces and materials, not just actions
- For multi-shot within a single generation, use "lens switch" between scene descriptions
- Append compact aesthetic + audio exclusion suffix
- **Important:** Seedance does NOT support `elements`, `scene_image_urls`, or `end_image_url`
### Elements-Capable Models (Kling O3 R2V — budget-overflow fallback, 7+ characters)
- Replace character names in descriptions with `@Element1`, `@Element2` tokens via regex
- Dialogue speaker uses element ref: `@Element1 (voice: low contralto...) says nervously: "..."`
- Scene image refs added as `Scene style references: @Image1, @Image2.`
- Append full aesthetic string + audio exclusion suffix (e.g., `No background music. Only generate dialogue, ambient sound, and sound effects.`)
### Reference-Image Models Without Tags (Vidu Q3)
- Use standard character names in prompt text (no element or image tokens)
- Attach `reference_image_urls` at the API layer (up to 4 flat images)
- Dialogue uses character name directly with voice description
### Text-Only Models (Kling V3 Pro, Veo 3.1)
- Inject full character descriptions including voice descriptions inline in the prompt
- No reference images attached (silently skipped)
- Structure: camera term first, then description, then dialogue with delivery cues
- Veo requires `resolution: '720p'`; Kling does NOT accept `resolution`/`aspect_ratio` (derived from input image)
### Native Multi-Shot (default: Seedance R2V Enhanced with `Lens switch.`)
**Multi-shot units render on `seedance-2-0-enhanced-reference-to-video` by default (2026-08-05)** — the same reference-first lane as singles. `buildMultiShotPrompt()` produces ONE Seedance generation covering the window's beats:
1. **Identity declarations up front** from the unit's @Image slot plan: `@Image1 is Bob — wearing …`
2. **Role clauses** for the non-character slots (blocking plate, location angles) — identical clause text to singles
3. **Per-beat blocks** labeled `Shot N (Xs):` with camera, description (names → `@ImageN`), authored `Blocking:` restated per beat (rule 49), and `[@ImageN, voice, delivery]: "line"` dialogue
4. **Literal `Lens switch.` lines** between beats (rule 21)
5. **Geometry hold** pinned to the plate (or first location angle): no mirroring/swapping across the internal cuts
6. Compact aesthetic + audio-exclusion suffix; ≤2500 chars
The unit renders in **pure reference mode** — `reference_image_urls` pushed in slot-plan order, no `image_url` start frame — and takes voice-donor `reference_audio_urls` (@AudioN) for its dialogue speakers. Grouping windows never span locations (one slot plan per generation).
### Legacy Multi-Shot Format (Kling 3.0 — explicit override only)
Used only when `videoDefaults.multiShotModel` is explicitly set to a non-@Image-tag model (e.g. `kling-o3-pro-image-to-video`, which has NO reference support — identity is prompt-text only).
**Prompt structure (per [Kling 3.0 Prompting Guide](https://blog.fal.ai/kling-3-0-prompting-guide/)):**
1. **Define subjects up front** with `@Element` refs and traits — Kling locks these across all shots
2. **State shot count and continuity instruction**: `"N-shot continuous sequence. Lock face, wardrobe, and environment across all shots."`
3. **Label each shot** as `Shot N (Xs):` with cinematic camera direction
4. **Dialogue format**: `[@Element1, voice description, delivery]: "dialogue line"` — bind dialogue to character actions
5. **Temporal separators**: `Immediately, cut to:` between shots for hard cuts
6. **Compact aesthetic** at the end — keep under 2500 chars total
**Example (2-shot dialogue with cut):**
```
@Element1 is SeehRov Kire: early 40s, lean, sandy-brown hair, pale blue-grey eyes.
Wearing ochre-sand linen overcoat, leather chest harness.
2-shot continuous sequence. Lock face, wardrobe across both shots.
Shot 1 (7s): Wide shot at a stone entrance.
@Element1 stands in the threshold facing the desert.
[@Element1, low measured baritone, quietly intense]: "I wrote the code that created a deity."
Sound of desert wind.
Immediately, cut to:
Shot 2 (6s): Extreme close-up on @Element1's face.
Golden amber sunlight from the side.
[@Element1, low measured baritone, voice hardening]: "Now I push the commit that destroys one."
```
**Key advantages over concatenated singles:**
- **Consistent identity** across the cut (same generation maintains character appearance)
- **Native audio continuity** (ambient sound and dialogue flow naturally)
- **Single API call** instead of multiple queue/poll cycles
**Grouping criteria:** Any consecutive shots with overlapping characters, no establishing/insert shots, total duration ≤ 15s. The planner greedily selects the largest valid window (up to 6).
### Environment Adaptation (All Models)
When a shot is set in a bright daytime environment while the project's default aesthetic is dark/moody:
1. **Image prompts:** Strip dark/rain-related terms from the aesthetic string; add anti-rain negative prompts
2. **Video prompts:** Append `Bright daytime scene, natural light, no rain.`
3. **Multi-edit (character fix):** Add `BRIGHT DAYTIME scene. Do NOT darken, no rain` instruction
4. **Multi-edit (style match):** Add `Keep bright warm lighting. Do NOT add rain, dark skies`
This prevents the project's default aesthetic from contaminating scenes with different lighting conditions.
## 7. Format-Specific Routing: Talk Shows, Interviews, Panels
For formats with **frequent speaker cuts** (talk shows, interviews, debate panels, podcasts), temporal continuity between shots matters less than character identity. Apply these overrides:
### All Character Shots Must Be R2V Singles
- **Never group shots with different speakers into multi-shot units.** Even on the reference-first Seedance multi-shot default, a unit builds ONE slot plan for the window — disjoint speakers mean the plan can't anchor everyone correctly. (On the legacy Kling override the failure is total: `kling-o3-pro-image-to-video` has NO `elements` or `reference_image_urls`, so characters lose all identity anchoring.)
- **Set `mustStaySingle: true`** on all shots in talk show scripts, or ensure the generation planner only groups shots that share the same characters.
- **Every character shot uses R2V** — `seedance-2-0-reference-to-video` for 1-2 character shots (flat `reference_image_urls` with `@Image` tags), auto-fallback to `kling-o3-standard-reference-to-video` for 3+ characters (structured `elements`).
### Multi-Shot Grouping Rules for These Formats
The `hasOverlappingCharacters` check requires **pairwise character overlap** between consecutive shots. Shots that cut between different speakers (host → guest, speaker A → speaker B) have zero overlap and must be separate.
Valid multi-shot grouping examples:
- Shot A (Host) → Shot B (Host) → OK, same character
- Shot A (Host + Guest) → Shot B (Host) → OK, host overlaps
Invalid grouping (now prevented):
- Shot A (Host only) → Shot B (Guest only) → BLOCKED, no overlap
- Shot A (Guest A) → Shot B (Guest B) → BLOCKED, no overlap
### Duration Validation
Seedance 2.0 (now the default for both atmosphere and character shots) accepts **4s, 5s, 8s, 10s, 12s, or 15s** — not all integers. Veo 3.1 (legacy) only accepts 4s/6s/8s. The video queue function auto-snaps invalid durations to the nearest valid value. When scripting shots, use 4s minimum instead of 3s.
## 8. Anti-Patterns and Learned Routing Failures
### API Errors
- **Sending `elements` to Seedance R2V:** Seedance does NOT support structured `elements`. Only `reference_image_urls` (flat array, up to 4). The harness gates this via `supportsElements: false` in the model spec.
- **Sending `scene_image_urls` to Seedance R2V:** Not supported. Scene context must be described in the prompt text only.
- **Using `@Element` tags in Seedance prompts:** Seedance uses `@Image1`, `@Image2` tags for reference images. The `useImageTags` flag in `ModelResolution` controls which tag system the prompt builder uses.
- **Sending `elements`/`reference_image_urls` to Kling V3 Pro or Kling O3 Pro (non-R2V):** Returns 400. Only R2V models support these params.
- **Sending `resolution`/`aspect_ratio` to Kling image-to-video models:** Returns 400. These are derived from the input image automatically.
- **Omitting `aspect_ratio` from R2V models:** Both Seedance R2V and Kling O3 R2V require `aspect_ratio`. If omitted, it defaults to `16:9` in code. Always pass `aspect_ratio` explicitly.
- **Sending `image_references`/`image_1` to `nano-banana-pro`:** Returns 400. The generation model does not accept reference payloads at all.
- **Sending invalid durations:** Seedance 2.0 accepts 4s/5s/8s/10s/12s/15s. Veo 3.1 accepts 4s/6s/8s. Duration auto-snap corrects this.
- **Sending `2K` to MiniMax H3 Max, or `768P` to plain MiniMax H3:** The two families share a name and invert on resolution. H3 is 2K-only; H3 Max and H3 Max Turbo top out at 768P and reject 2K. `video-generator.ts` pins each family, and the `-max` branch has to stay above the `minimax-h3` substring match or every H3 Max render 400s.
- **Reaching for `minimax-h3-max-turbo-reference-to-video`:** It doesn't exist ("Specified model not found"). Turbo has no R2V lane; identity shots route to `minimax-h3-max-reference-to-video`.
- **Sending `audio` to any MiniMax H3 family model:** `audioConfigurable: false` — audio is always generated and the field must be omitted from the body, not set to `true`. (`queueVideo` now strips it automatically for these models.)
- **Sending `camera_trajectory` to anything but `minimax-h3-max-multi-angle`:** the strict queue schema 400s with "Unrecognized key(s) in object: 'camera_trajectory'". On multi-angle itself, azimuth travel over 32 turns 400s with "Camera azimuth travel must not exceed 32 full turns" — `validateCameraTrajectory` catches both before you pay.
- **Reference images below 300x300:** R2V models reject `reference_image_urls` and `elements` images smaller than 300x300 pixels. Never downscale character references below this threshold.
- **Seedance + non-seedream face images (no longer an issue, 2026-07):** Venice removed the restriction that Seedance 2.0 only accepts face-bearing input images from `seedream-v5-lite` / `seedream-v5-lite-edit`. Any image family now works for face-bearing inputs, so there's nothing to pair, reroute, or launder — the pre-flight gate is a no-op.
### Visual Contamination
- **Frame chaining from dark scene to bright scene:** The dark tone from the last frame bleeds into the bright scene. Use the panel image instead of chaining when the environment changes.
- **Multi-edit with dark character references re-darkening bright panels:** Character reference images shot in dark environments cause the edit model to match the overall tone, not just the face. Use environment adaptation instructions to preserve brightness.
- **Project aesthetic contaminating daytime scenes:** A dark/rainy default aesthetic bleeds into bright interiors unless explicitly stripped via environment adaptation.
### Identity Failures
- **Grouping different-character shots into multi-shot units:** A multi-shot unit builds ONE reference slot plan for the window — disjoint characters can't all anchor correctly (and on the legacy Kling override there are no references at all). Always verify pairwise character overlap before grouping.
- **Frame chaining when new character enters:** The video model invents the new character's appearance from nothing. Always use the panel image (which was refined against character references) as the start frame when a new character appears.
- **Multi-edit with more than 2 character references:** The multi-edit endpoint accepts max 3 images total (base + 2 refs). Exceeding this drops references silently.
- **Sequential action in image descriptions:** Causes comic-panel layouts instead of single frames. Separate the single-frame panel description from the full video action description.
- **Vague body orientation:** Produces twisted poses. Always specify full-body direction explicitly (e.g., "seen entirely from behind", "facing camera directly").
### Prompt-Style Mismatches
- **Over-directing a simple-prompt model:** Hand-writing blocking, per-beat camera terms, `Lens switch.` lines, or geography-hold clauses into a MiniMax H3 Max prompt flattens the result — it stages its own coverage and the clauses fight it. The prompt builder strips these automatically; don't re-add them via authored shot fields expecting them to help.
- **Padding an H3 Max prompt to clear the motion gate:** The Creator's `produce_shots` money gate normally demands 12+ words and motion vocabulary. For simple-prompt models it only asks for a stated subject and setting. Inflating a short, correct prompt to satisfy the old bar is the failure, not the fix.
### Style Consistency Failures
- **Aesthetic description buried at end of prompt:** The model commits to a rendering style before reaching the style instructions, causing inconsistency between angles/shots. Always front-load style with a `STYLE:` prefix and add a `STYLE REMINDER:` suffix.
- **Low cfg_scale for character references:** Using `cfg_scale: 7` gives the model too much freedom, causing style drift between angles (e.g., front is cartoon, profile is photorealistic). Use `cfg_scale: 10` for all character references and storyboard panels.
- **No anti-realism terms in negative prompt:** Without explicit `photorealistic, photograph, photo` in the negative prompt, stylized/illustration aesthetics drift toward realism on some angles.
- **Lighting mismatch between consecutive shots in same location:** Each panel generated independently can produce wildly different lighting for the same environment. Style-match later panels against earlier ones using multi-edit, or explicitly describe the established lighting.
### Multi-Edit Pitfalls
- **Multi-editing 16:9 close-ups destroys foreheads:** Multi-edit returns 1024x1024. Cropping to 16:9 removes ~25% from top and bottom. Close-up face shots lose foreheads and chins. For close-ups needing forehead detail (logos, headwear), generate from scratch with `nano-banana-pro` instead.
- **Establishing shots with silhouetted figures treated as empty:** Shots with `characters: []` trigger "no people" negative prompts, preventing silhouettes from appearing. Use `silhouetteCharacters` in the shot script for distant/back-turned figures that don't need R2V.
- **Logo/sigil descriptions misinterpreted:** Shorthand like "triple-V" or "VVV" produces random V-shapes. Always describe the actual geometric design (e.g., "two ornate skeleton keys crossed in an X with a chevron/book at top") or use the logo image file as a multi-edit reference.
## Troubleshooting
**"VENICE_API_KEY not set"** -- Set the environment variable or add it to your project's `.env` file.
**"Model not found"** -- Run `--list-models` to see available models. Use `--no-validate` for new/beta models not yet in the model list.
**Video stuck/timeout** -- Videos can take 1-5 minutes depending on model and duration. Use `--timeout 600` for longer videos.
**"requests" module not found** -- Install it: `pip3 install requests`