generating-videos · git:20260317.b3d2ddd · 2026-03-17 · sha256 259e1913e091ab1e
generating-videos git:20260317.b3d2dddA
Immutable. This exact content never changes and is served at /api/v1/blob/259e1913e091ab1e.
No tool to install first
claude plugin marketplace add https://markdownregistry.com/mp/a/art_dckiti7emeyr3y75/259e1913e091ab1e325630afe4ca5f5427af5eb15fa663912b25d3c6c729fb0d.json
Then /plugin install generating-videos@markdownregistry-dckiti7emeyr3y75 in Claude Code. It installs this exact version, the one graded A here, pinned to the commit it was graded at, and leaves it there.
---
name: generating-videos
description: Generates videos using Grok (preferred) and Google Veo 3.1 models. Supports text-to-video, image-to-video, first-last-frame, video extension, and reference images. AI视频生成、文生视频、图生视频、首尾帧生成、视频拓展。
---
# Video Generation
Generates videos using **Grok** (preferred) and **Google Veo 3.1** models.
## Model Selection Strategy
**CRITICAL**: Always prefer Grok unless the task requires Veo-specific features.
| Scenario | Use Model | Reason |
|----------|-----------|--------|
| Video ≤ 15s (text-to-video / image-to-video) | **Grok** (preferred) | Grok supports 1-15s directly, faster and cost-effective |
| Video 16-36s (continuous) | **Veo** | Requires Video Extension, Grok max is 15s |
| First-last-frame storyboard | **Veo** | Grok does not support first-last-frame |
| Reference images (style consistency) | **Veo** (generate-preview) | Grok does not support reference images |
| User explicitly requests Veo | **Veo** | User preference |
| User explicitly requests Grok | **Grok** | User preference |
## Language Rule
**CRITICAL**: Video prompts MUST be generated in:
1. **User's language** - If user writes in Chinese, generate Chinese prompts; if in English, generate English prompts
2. **User-specified language** - If user explicitly requests a specific language, use that language
---
## Grok Video Models (Preferred)
| Model | Speed | Quality | Max Duration | Use Case |
|-------|-------|---------|--------------|----------|
| grok-imagine-video | Fast | Good | 15s | Default - Most use cases |
### Grok Parameters
| Parameter | Values | Default | Description |
|-------------|-------------------------------------------------|---------|-------------|
| duration | 1-15 | - | Video duration in seconds |
| resolution | 480p, 720p | 720p | Video resolution |
| aspectRatio | 1:1, 16:9, 9:16, 4:3, 3:4, 3:2, 2:3 | 9:16 | Video aspect ratio |
| imageUrl | string | - | Reference image URL for image-to-video |
### Grok Generation Modes
| Mode | Parameters | Description |
|------|-----------|-------------|
| Text-to-video | prompt only | Generate video from text description |
| Image-to-video | prompt + imageUrl | Generate video using a reference image |
### Grok Workflow
1. Call `generateVideoWithGrok` with prompt and optional parameters
2. Poll `getGrokVideoStatus` every 30 seconds
3. Maximum wait: 5 minutes
---
## Veo 3.1 Models (Advanced Features)
Use Veo when you need: video extension, first-last-frame, or reference images.
| Model | Speed | Quality | Video Extension | Reference Images | Use Case |
| ----------------------------- | ----- | ------- | --------------- | ---------------- | -------------------------------- |
| veo-3.1-fast-generate-preview | Fast | Good | Yes | No | Default Veo - Most use cases |
| veo-3.1-generate-preview | Slow | Better | Yes | Yes (max 3) | Best quality + all features |
| veo-3.1-fast-generate-001 | Fast | Good | No | No | Simple generation (no extension) |
| veo-3.1-generate-001 | Slow | Better | No | No | Higher quality (no extension) |
**Veo Model Selection**:
- **Default Veo**: Use `veo-3.1-fast-generate-preview` (supports video extension)
- **Use non-preview models when**: User explicitly requests and doesn't need video extension
- **Use non-fast models when**: User explicitly requests higher quality
### Veo Parameters
| Parameter | Values | Default | Description |
| -------------- | ----------------- | ------- | ---------------------------------------- |
| duration | 4, 6, 8 | 8 | Video duration in seconds |
| resolution | 720p, 1080p, 4000 | 720p | Video resolution (1080p/4K takes longer) |
| aspectRatio | 16:9, 9:16 | 9:16 | Video aspect ratio (vertical by default for social media) |
| negativePrompt | string | - | What to exclude from the video |
| seed | number | - | Random seed for reproducibility |
**Notes**:
- Video extension only supports 720p
- Reference images require 8-second duration
---
## Prompt Guide (applies to both Grok and Veo)
### Prompt Structure
`[Subject & Background] + [Action] + [Style] + [Camera] + [Atmosphere] + [Audio]`
### 1. Subject & Background (主题与背景)
Specify your main focus (object, person, animal) and environmental context.
**Examples**:
- "A young woman with long black hair wearing a red dress" / "一位长发黑发、身穿红裙的年轻女子"
- "White concrete apartment building with organic shapes and lush greenery" / "白色混凝土公寓楼,有机形状,绿植茂盛"
### 2. Action (动作)
Describe what the subject is doing—walking, running, transforming, etc.
**Examples**:
- "walks slowly towards the camera" / "缓缓走向镜头"
- "transforms from liquid to solid" / "从液态变为固态"
### 3. Style (风格)
Add aesthetic direction using keywords.
**Common Styles**:
- Film noir / 黑色电影
- Surrealism / 超现实主义
- Cyberpunk / 赛博朋克
- 3D cartoon animation / 3D卡通动画
- Cinematic documentary / 电影纪录片
- Japanese anime / 日系动漫
### 4. Camera Work (镜头)
Control perspective and movement.
**Camera Angles**:
- Aerial shot / 航拍镜头
- POV (Point of View) / 第一人称视角
- Close-up / 特写
- Wide angle / 广角
- Low angle / 低角度
- Bird's eye view / 鸟瞰
**Camera Movements**:
- Tracking shot / 跟踪镜头
- Pan left/right / 左右平移
- Dolly in/out / 推拉镜头
- Static shot / 固定镜头
### 5. Atmosphere (氛围)
Specify lighting and color mood.
**Examples**:
- "warm golden hour lighting" / "温暖的黄金时段光线"
- "cool blue tones" / "冷蓝色调"
- "neon-lit cyberpunk atmosphere" / "霓虹灯赛博朋克氛围"
- "foggy moonlight" / "雾蒙蒙的月光"
---
## Audio Generation Guide
Veo 3.1 natively generates synchronized audio. Include audio descriptions in your prompt.
### 1. Dialogue (对话)
Use **quotation marks** for specific speech.
**Example**:
```
A man in a suit stands at a podium, speaking confidently: "Welcome to the future of technology."
一位穿西装的男子站在讲台上,自信地说:"欢迎来到科技的未来。"
```
### 2. Sound Effects (音效)
Describe sounds explicitly.
**Examples**:
- "screeching tires" / "轮胎尖叫声"
- "crackling fire" / "噼啪作响的火焰"
- "thunder rumbling in the distance" / "远处雷声隆隆"
- "footsteps echoing in the hallway" / "走廊里回响的脚步声"
### 3. Ambient Noise (环境音)
Describe environmental soundscapes.
**Examples**:
- "busy city street with traffic and chatter" / "繁忙的城市街道,车流和人声"
- "peaceful forest with birds chirping" / "宁静的森林,鸟鸣声"
- "ocean waves crashing on the shore" / "海浪拍打海岸"
- "busy city street with traffic and chatter" / "繁忙的城市街道,车流和人声"
- "peaceful forest with birds chirping" / "宁静的森林,鸟鸣声"
- "ocean waves crashing on the shore" / "海浪拍打海岸"
## Generation Modes
Modes are auto-detected based on parameters:
| Mode | Parameters | Models Supported |
| ---------------- | -------------------------- | ----------------------------- |
| Text-to-video | prompt only | All |
| Image-to-video | prompt + image | All |
| First-last-frame | prompt + image + lastFrame | All |
| Video extension | prompt + video | Preview models only |
| Reference images | prompt + referenceImages | veo-3.1-generate-preview only |
---
## Video Extension
Extend existing Veo-generated videos.
**IMPORTANT**: Each extension returns a **COMPLETE video** (initial + all extensions combined). NO concatenation needed - the API automatically merges the extended content into a single video file.
### Limitations
- **Models**: Only preview models (`veo-3.1-generate-preview`, `veo-3.1-fast-generate-preview`)
- **Extension length**: ~7 seconds per extension
- **Max extensions**: 4 times
- **Max total duration**: 36 seconds (8s initial + 4×7s extensions)
- **Resolution**: 720p only
- **Aspect ratio**: 16:9 or 9:16 only
### Workflow
1. Generate initial video with preview model
2. Call `generateVideoWithVeo` with `video` parameter (prefer GCS URI `gs://bucket/path` for efficiency)
3. Repeat up to 4 times to reach 36 seconds max
**Note**: When extending videos, prefer using GCS URI over HTTP URL for better performance (no download/base64 conversion needed).
- **Models**: Only preview models (`veo-3.1-generate-preview`, `veo-3.1-fast-generate-preview`)
- **Extension length**: ~7 seconds per extension
- **Max extensions**: five times
- **Max total duration**: 37 seconds
- **Resolution**: 720p only
- **Aspect ratio**: 16:9 or 9:16 only
## Workflow
### Grok Video Generation (Preferred)
1. Call `generateVideoWithGrok` with prompt and optional parameters
2. Poll `getGrokVideoStatus` every 30 seconds
3. Maximum wait: 5 minutes
### Veo Video Generation
1. Call `generateVideoWithVeo` with prompt and optional parameters
2. Poll `getVeoVideoStatus` every 30 seconds
3. Maximum wait: 10 minutes
### First-Last-Frame Generation (Veo only)
1. Generate keyframes with `generateImage`
2. Call `generateVideoWithVeo` with `image` (first frame) and `lastFrame`
3. Poll `getVeoVideoStatus` until completed
---
## Duration-Based Model Selection Strategy
**CRITICAL**: For videos ≤ 15 seconds, ALWAYS prefer **Grok direct generation**. Only use Veo Extension for videos longer than 15 seconds.
### When to Use Each Approach
| Target Duration | Recommended Approach |
| --------------- | ---------------------------------------- |
| ≤ 15 seconds | **Grok direct generation** (preferred, faster and cost-effective) |
| 16-36 seconds | Veo Video Extension (dynamic initial + N extensions) |
| > 36 seconds | First-Last-Frame Storyboard |
### Why Grok for ≤ 15s?
- **Faster**: Grok generates directly without extension polling
- **Cost-effective**: Single generation call vs initial + extensions
- **Simpler workflow**: No multi-step extension process needed
- **Full duration support**: Grok supports 1-15 seconds natively
### Veo Extension (for 16-36s only)
**Dynamic Initial Duration Calculation**:
**Parameters**:
- Initial duration options: 4s, 6s, 8s
- Each extension adds: ~7 seconds
- Maximum extensions: 4 times
- Maximum total duration: 36 seconds (8s + 4×7s)
**Calculation Formula**:
1. Extension count `n` = ⌈(target - 8) / 7⌉
2. Initial duration = target - (n × 7)
3. If initial < 4, then n += 1, recalculate initial
**Reference Table**:
| Target | Initial | Extensions | Actual |
|--------|---------|------------|--------|
| 18s | 4s | 2 | 18s |
| 22s | 8s | 2 | 22s |
| 27s | 6s | 3 | 27s |
| 29s | 8s | 3 | 29s |
| 36s | 8s | 4 | 36s |
### Extension Workflow
1. Calculate optimal initial duration using the formula above
2. Generate initial video with preview model (720p) using calculated duration
3. Extend N times with `generateVideoWithVeo(video=<previous_video_gcs_uri_or_url>)`
4. Each extension returns a **COMPLETE accumulated video** (no concatenation needed)
**IMPORTANT**: The Gemini video extension API returns a complete video file containing the initial video plus all extensions merged together. You do NOT need to concatenate segments - just use the final extended video directly.
**Note**: Prefer using GCS URI (`gs://bucket/path`) for video extension - it's more efficient than HTTP URL.
**Example**: Target 22 seconds
- n = ⌈(22 - 8) / 7⌉ = 2
- Initial = 22 - (2 × 7) = 8s
- Result: 8s initial + 2 extensions = 22s ✓
**Note**: Extension uses the video URL directly - no need to call `uploadAndGetVid`.
---
## Long Videos Strategy
### Approach 1: First-Last-Frame Storyboard
**CRITICAL**: This workflow has strict sequential dependencies. Do NOT skip steps.
Use Veo's first-last-frame for precise visual control in multi-segment videos.
**Frame Sharing Principle**: `N video segments require N+1 keyframes`
**Workflow** (MUST follow in order):
**Step 1: Generate ALL keyframes FIRST**
- Generate N+1 keyframes with `generateImage`
- Pass previous frames as `imageUrls` for style consistency
- Example: 6 videos need 7 keyframes (frame1 to frame7)
**Step 2: Generate videos with shared frames**
- Video 1: `generateVideoWithVeo(image=frame1, lastFrame=frame2)`
- Video 2: `generateVideoWithVeo(image=frame2, lastFrame=frame3)` ← frame2 shared!
- ... continue for all segments
- Videos CAN be generated in parallel AFTER all keyframes are ready
**Step 3: Concatenate**
- Load `editing-videos` skill, then use `submitDirectEditTask` to combine all segments
- **CRITICAL: NO transitions** - shared keyframes ensure seamless visual continuity
### Approach 2: Video Extension
For continuous long videos without cuts.
**Workflow**:
1. Generate initial 8-second video with preview model (720p)
2. Extend with `generateVideoWithVeo(video=<previous_video_gcs_uri_or_url>)`
3. Repeat up to 4 times (max 36 seconds total)
**Note**: Prefer using GCS URI (`gs://bucket/path`) for video extension - it's more efficient than HTTP URL.
**Limitations**:
- 720p only
- 16:9 or 9:16 only
- Must use preview models
---
## Video Concatenation Guide
When concatenating multiple generated video segments, use `submitDirectEditTask` with the Track structure.
### Prerequisites
1. Load the `editing-videos` skill first
2. Upload each video segment using `uploadAndGetVid` to get VID
### Concatenation Workflow
**Step 1**: Upload all video segments
```
For each segment URL:
uploadAndGetVid(url) → vid://segment_N, duration_N, width, height
```
**Step 2**: Build Track structure
**CRITICAL**: Choose the correct method based on how videos were generated:
#### For First-Last-Frame Videos - NO TRANSITIONS
⚠️ **MUST use direct concatenation. DO NOT add transitions.**
When videos are generated using Veo first-last-frame with shared keyframes:
```json
{
"Canvas": { "Width": 1920, "Height": 1080 },
"Track": [[
{ "Type": "video", "Source": "vid://segment_1", "TargetTime": [0, 8000] },
{ "Type": "video", "Source": "vid://segment_2", "TargetTime": [8000, 16000] },
{ "Type": "video", "Source": "vid://segment_3", "TargetTime": [16000, 24000] }
]]
}
```
**Why no transitions?** The shared keyframes between segments already ensure seamless visual continuity.
#### For Independent Segments - Optional Transitions
When videos are independently generated (not using shared keyframes), transitions can help smooth the connection:
```json
{
"Canvas": { "Width": 1920, "Height": 1080 },
"Track": [[
{
"Type": "video",
"Source": "vid://segment_1",
"TargetTime": [0, 8000],
"Extra": [{ "Type": "transition", "Source": "1182376", "Duration": 500 }]
},
{
"Type": "video",
"Source": "vid://segment_2",
"TargetTime": [7500, 15500],
"Extra": [{ "Type": "transition", "Source": "1182376", "Duration": 500 }]
},
{ "Type": "video", "Source": "vid://segment_3", "TargetTime": [15000, 23000] }
]]
}
```
**Key Points**:
- Time unit is **milliseconds** (1s = 1000ms)
- For transitions, overlap TargetTime by the transition Duration
- Last segment does NOT need a transition
- **First-last-frame videos MUST use direct concatenation (no transitions)**
**Step 3**: Submit and poll
```
submitDirectEditTask(Canvas, Track) → taskId
wait 90 seconds
poll getVideoEditTaskStatus(taskId) every 30 seconds until completed
```
---
## Output Requirements
**MANDATORY**: After generating videos, you MUST output ALL video URLs clearly.
### Single Video Output
```
**Generated Video**:

Video URL: url
```
### Multiple Videos Output (Batch/Storyboard)
```
All N video segments completed!
| # | Status | Preview | URL |
|---|--------|---------|-----|
| 1 | ✅ |  | url1 |
| 2 | ✅ |  | url2 |
| ... | ... | ... | ... |
**All Video URLs**:
1. url1
2. url2
...
```
### After Concatenation
```
**Final Video** (concatenated from N segments):

Final Video URL: final_url
```
**IMPORTANT**: Never end a video generation task without explicitly listing all video URLs.