parallel git:20260930.72da14aA

Immutable. This exact content never changes and is served at /api/v1/blob/76d033d4cdf49c87.

---
name: parallel
description: "Parallel execution patterns for montaj — load when workflow has multiple clips or foreach steps"
step: false
---

# Parallel Execution

**Important:** Claude Code's Bash tool executes sequentially — multiple Bash calls in the same turn queue one after another. They do NOT run in parallel. Use the patterns below for true parallelism.

## Step-level — same step, multiple clips

For `foreach` steps on independent clips, use a **single Bash call with background jobs**:

```bash
out0=$(curl -s -X POST http://localhost:3000/api/steps/transcribe \
  -H "Content-Type: application/json" \
  -d '{"input": "/path/clip0.MOV", "model": "base.en"}') &
out1=$(curl -s -X POST http://localhost:3000/api/steps/transcribe \
  -H "Content-Type: application/json" \
  -d '{"input": "/path/clip1.MOV", "model": "base.en"}') &
# ... repeat for each clip
wait
echo "$out0"; echo "$out1"  # parse results after all complete
```

Use for any independent `foreach` step: `waveform_trim`, `transcribe`, `rm_fillers`, `probe`, `snapshot`.

**Do NOT use background jobs for encoding steps.** See Encoding steps below.

## Encoding steps — `materialize_cut`, `remove_bg`

Encoding steps are memory-intensive per instance (full libx264 encode + frame buffers). Fanning them out unbounded across N clips will exhaust memory on 4K footage. Use batch mode with the built-in concurrency cap instead of background jobs:

```bash
# materialize_cut — batch mode, 2 concurrent encodes (default)
montaj materialize-cut --inputs clip0_spec.json clip1_spec.json clip2_spec.json
# → JSON array of output paths

# raise to 3 only on machines with 32GB+ RAM and 1080p or smaller footage
montaj materialize-cut --inputs clip0.MOV clip1.MOV --workers 3

# remove_bg — always use --inputs; runs on the CPU, one clip at a time
montaj remove-bg --inputs clip0_cut.mp4 clip1_cut.mp4 clip2_cut.mp4
# → JSON array of {nobg_src, nobg_preview_src} objects
```

**`remove_bg` is long-running** (minutes per clip, CPU only). Always run it in the background when using the Agent tool or HTTP API so the agent remains responsive.

## `waveform_trim` native batch

Preferred over background jobs — pass all clips in a single call:

```bash
# HTTP
curl -s -X POST http://localhost:3000/api/steps/waveform_trim \
  -H "Content-Type: application/json" \
  -d '{"inputs": ["/path/clip0.mp4", "/path/clip1.mp4", "/path/clip2.mp4"]}'
# → returns JSON array of trim specs: [{"input": "...", "keeps": [...]}, ...]

# CLI
montaj waveform-trim clip0.mp4 clip1.mp4 clip2.mp4
# → JSON array of trim specs printed to stdout
```

## Clip-level — multiple steps per clip (swarm)

When each clip needs 3+ sequential per-clip steps, background jobs get unwieldy. Use **one subagent per clip**:

1. Identify the fan-out point: every step declared `foreach: clips`
2. Identify the fan-in point: the first step with no `foreach` that actually consumes every clip's output — an agent-authored task in the clip-based workflows (`select-takes`, `overlay`). Not every non-`foreach` step is a fan-in: `broll`'s whole `vo_*` chain is non-`foreach` because it runs on the voiceover alone, in parallel with the footage pass
3. Spawn one subagent per clip — each receives: clip path, steps to run, project id, editing prompt
4. Cap at **4 concurrent clip agents** to avoid resource contention
5. Wait for all subagents to complete, collect output paths
6. Fan in: hand the collected paths to the non-`foreach` step and continue

There is no join step. Nothing concatenates the per-clip outputs into one file: the surviving keeps become `tracks[0]` items with their own `inPoint`/`outPoint`, and the render engine assembles them in a single pass at the end.

Use when a clip needs 3+ sequential steps — subagent coordination overhead is worth it beyond that threshold.

## Dependency waves

Workflow steps declare `needs: [step_ids]`. Identify waves of ready steps (all `needs` met) and execute each wave with the appropriate parallel pattern before moving to the next.

**Example — default workflow:**

| Wave | Steps | Pattern |
|------|-------|---------|
| 1 | `probe`, `snapshot`, `waveform_trim` (per clip) | Bash `&` or waveform_trim batch |
| 2 | `rm_nonspeech` (per clip) | Bash `&` |
| 3 | `transcribe` (per clip) | Bash `&` |
| 4 | `select-takes` | Sequential — the fan-in; needs every clip's transcript |
| 5 | `rm_fillers` (per clip) | Bash `&` |
| 6 | `transcribe_final` (per clip) | Bash `&` |
| 7 | `overlay` | Sequential |

Assembly is not a wave. The render engine reads the finished `tracks` and encodes once, at render time.