video-frames git:20260721.3199962A

Immutable. This exact content is served forever at /api/v1/blob/b995f30954efbc93.

---
name: video-frames
description: This skill should be used when the user asks to "extract frames", "analyze video frames", "get screenshots from videos", "run vision analysis on videos", "analyze on-screen text in videos", "create frame grids", or needs to extract and visually analyze frames from downloaded video files.
argument-hint: "[optional: path to video directory or metadata.json]"
allowed-tools: ["Read", "Write", "Edit", "Bash", "Glob", "Grep", "Agent", "AskUserQuestion"]
---

# Frame extraction and vision analysis

Extract frames from video files at regular intervals, create 3x3 grid composites for efficient viewing, and run vision analysis to catalog on-screen text, settings, and visual elements.

## Prerequisites

```bash
ffmpeg -version       # Frame extraction
python -c "from PIL import Image; print('Pillow OK')"  # Grid compositing
```

Install if missing: `pip install Pillow`

## Workflow

### Step 1: Configure extraction parameters

Ask the user or use defaults:

| Parameter | Default | Description |
|-----------|---------|-------------|
| Interval | 3 seconds | One frame every N seconds |
| Max width | 1920px | Scale down wider frames |
| Quality | 95% JPEG | `-q:v 2` in ffmpeg |
| Grid size | 3x3 | Frames per composite grid |
| Grid cell size | 640x360 | Pixels per cell in the grid |

### Step 2: Extract frames with ffmpeg

For each video in metadata.json:

```bash
ffmpeg -i {video_path} \
  -vf "fps=1/{interval},scale='min({max_width},iw)':-1" \
  -q:v 2 -start_number 0 \
  {frames_dir}/{platform}/{video_id}/frame_%04d.jpg \
  -y -loglevel error
```

Frames are sequentially numbered: `frame_0000.jpg` = 0s, `frame_0001.jpg` = 3s, `frame_0002.jpg` = 6s, etc.

**Windows note:** Do not rename frames after extraction. `Path.rename()` fails on Windows when the target exists. Use sequential numbering with a documented interval mapping instead.

Skip videos that already have frames extracted.

### Step 3: Create 3x3 grid composites

Grid composites let Claude analyze 9 frames at once and see visual transitions between them.

```python
from PIL import Image

GRID_SIZE = 3
CELL_W, CELL_H = 640, 360

frames = sorted(frame_dir.glob("frame_*.jpg"))
for batch_start in range(0, len(frames), GRID_SIZE * GRID_SIZE):
    batch = frames[batch_start:batch_start + 9]
    grid = Image.new("RGB", (CELL_W * 3, CELL_H * 3), (0, 0, 0))
    for i, frame_path in enumerate(batch):
        row, col = i // 3, i % 3
        img = Image.open(frame_path)
        img.thumbnail((CELL_W, CELL_H))
        x = col * CELL_W + (CELL_W - img.width) // 2
        y = row * CELL_H + (CELL_H - img.height) // 2
        grid.paste(img, (x, y))
    grid.save(grid_dir / f"grid_{batch_start:04d}.jpg", quality=85)
```

Save grids to `frame-grids/{platform}/{video_id}/`.

### Step 4: Vision analysis

Read grid composites using the Read tool and write structured analysis JSON per video.

**Sampling strategy:** For efficiency, read the first, middle, and last grid per video. This covers the opening, core content, and closing of each video with ~3 Read calls per video instead of dozens.

For each grid, note:
- **On-screen text:** All visible text — captions, subtitles, headlines, lower-thirds, URLs, graphics text, watermarks
- **Setting:** Where was this filmed? (office, street, studio, subway, press room, etc.)
- **Visual elements:** Key objects, people, graphics, charts visible
- **Presentation style:** Formal/casual, handheld/tripod, documentary/direct-to-camera, etc.

**Output format** per video at `frame-analysis/{platform}/{video_id}.json`:

```json
{
  "video_id": "...",
  "platform": "...",
  "frames": [
    {
      "grid": "grid_0000.jpg",
      "timestamp_range": "0s-24s",
      "on_screen_text": ["text1", "text2"],
      "setting": "NYC subway station",
      "visual_elements": ["podium", "microphones"],
      "presentation_style": "formal press conference"
    }
  ],
  "summary": {
    "dominant_setting": "...",
    "text_overlay_types": ["captions", "lower-thirds"],
    "visual_themes": ["governance", "community"]
  }
}
```

**Parallelization:** Dispatch one subagent per platform for vision analysis. Each agent reads its platform's grids and writes the JSON files independently.

### Step 5: Verify and report

Report:
- Total frames extracted
- Total grids created
- Videos with vision analysis completed
- Any failures

Commit frame-analysis JSON files (not the frames or grids themselves — those are gitignored).

## Key lessons

- **3x3 grids are essential:** Reading individual frames is too slow and lacks temporal context. Grid composites reduce Read calls by 9x and show visual transitions.
- **Sample first/middle/last:** For 76 videos, full grid analysis means 700+ images. Sampling 3 grids per video (~228 total) gives good coverage.
- **Parallel subagents:** Dispatch one agent per platform for vision analysis. They don't conflict since each writes to a separate platform directory.
- **Sequential numbering over renaming:** On Windows, avoid renaming frames to timestamp-based names. Sequential numbering with a documented interval mapping is simpler and avoids filesystem errors.