git:20260417.89d056b to git:20260428.4ab9c84

57 added, 203 removed. Audit A to A.

---

name: lai-diarize

- description: Use when user needs speaker identification in audio/captions. Detects who speaks when, assigns speaker labels, and optionally infers speaker names from context. Use this skill whenever the user has a podcast, interview, meeting recording, or any multi-speaker audio and wants to know who said what. Also triggers on "diarize", "speaker detection", "说话人识别", "who is speaking", "add speaker labels", "区分说话人", "identify speakers", "which speaker said this", even if they just say "this podcast has two people, I need their names labeled".

- allowed-tools: Read, Bash(lai:*)

+ description: Identify speakers ("who said what") in aligned captions via pyannote.audio. Real speaker names come from the agent's own reasoning over transcript + context (default), with a CLI-LLM fallback for headless runs. Trigger on multi-speaker content (podcasts, interviews, meetings) or phrases like "diarize", "speaker detection", "说话人识别", "区分说话人", "label the speakers". Requires aligned captions — run `/lai-align` first.

---

# Speaker Diarization

- Identify and label speakers in aligned captions. Uses pyannote.audio for

- speaker detection and optional LLM-powered speaker name inference.

- **CLI syntax note**: LattifAI uses `nemo_run` configuration syntax. Options use

- `key=value` format (e.g., `diarization.infer_speakers=true`), NOT traditional

- flags. This applies to all `lai` subcommands.

- **Requires**: Aligned captions as input (run `/lai-align` first).

- ## When to Use

- - Multi-speaker content (podcasts, interviews, meetings, panel discussions)

- - Need to add speaker labels to aligned captions

- - Want to identify who said what

- - Speaker-aware downstream tasks (translation, summarization)

- ## When NOT to Use

- - Single-speaker content (narration, audiobooks) -- speaker label not needed

- - No aligned captions yet -- run `/lai-align` first

- - Only need timing fixes -- use `/lai-align`

- - Need transcription -- use `/lai-transcribe`

- ## Prerequisites

- - Aligned captions with timing (from `/lai-align`)

- - LattifAI API key (`lai auth whoami`)

- - For speaker name inference: LLM backend configured

- ```bash

- lai config set diarization.infer_speakers true

- lai config set diarization.llm.model_name gemini-2.5-flash

- ```

- ## CLI Usage

- ### Basic Diarization

- ```bash

- lai diarize run audio.mp3 aligned.json diarized.json

- ```

- Shortcut:

+ > **Preferred model: Claude Sonnet** (cost-efficient for agent-driven naming). This skill runs on whatever model is active in the parent session — any Claude model works; no hard switch. Sonnet has no 1M-context variant, so if the parent session is Opus[1M], continuing on Opus is normal (avoids a no-op model swap).

- ```bash

- lai-diarize audio.mp3 aligned.json diarized.json

- ```

+ Adds `speaker` labels to aligned captions. Speaker **detection** (who speaks when) is always CLI-based via pyannote.audio; speaker **naming** (who each one is) is agent-driven by default.

- ### With Context (Speaker Hints)

+ ## Basic Command

- Provide context via a `.meta.md` file or inline string for better speaker

- identification:

+ `<base>` = source media stem (e.g. `podcast` from `podcast.mp3`) or YouTube ID. Files all land in the current directory:

```bash

- # From meta.md file (YAML frontmatter with speakers, title, channel)

- lai diarize run podcast.mp3 aligned.json diarized.json \

- context=episode.meta.md

- # Inline context string

- lai diarize run podcast.mp3 aligned.json diarized.json \

- context="Host: Alice Chen (tech journalist), Guest: Bob Smith (AI researcher)"

+ lai diarize run podcast.mp3 podcast.aligned.json podcast.diarized.json

+ # shortcut:

+ lai-diarize podcast.mp3 podcast.aligned.json podcast.diarized.json

```

- ### With Speaker Name Inference

+ Output labels detected speakers as `SPEAKER_00`, `SPEAKER_01`, …

- LLM analyzes the conversation to infer speaker names from context:

+ Speaker count is auto-detected. Override only when auto-detection is clearly wrong:

- ```bash

- lai diarize run podcast.mp3 aligned.json diarized.json \

- diarization.infer_speakers=true

- ```

+ - `diarization.num_speakers=3` — exact count (when known)

+ - `diarization.min_speakers=N` / `diarization.max_speakers=N` — bound the search

- Requires an LLM backend:

- ```bash

- lai config set diarization.llm.model_name gemini-2.5-flash

- lai config set GEMINI_API_KEY your-key

- ```

+ ## Giving Speakers Real Names

- ### Specify Number of Speakers

+ ### Agent-driven (default)

- If you know the exact speaker count:

+ After the basic command finishes, the agent reads `diarized.json` together with any available context and rewrites `SPEAKER_XX` in-place with real names.

- ```bash

- lai diarize run meeting.mp3 aligned.json diarized.json \

- diarization.num_speakers=3

- ```

+ Signals the agent uses:

- ## Parameters

+ 1. **Explicit context** the user provides in the conversation

+ 2. **`meta.md`** beside the source (YAML frontmatter, format below)

+ 3. **Transcript evidence** — self-introductions ("I'm Alice…"), mutual addressing ("thanks, Bob"), host/guest dynamics, topical expertise

+ 4. **Existing inline labels** in the source text (`[Alice]`, `>> Bob:`, `SPEAKER_01:`) — preserved by the CLI and matched by the agent

- |-----------|-----------|---------|-------------|

+ Process:

- ## Context File Format (meta.md)

+ 1. Read `diarized.json` — collect unique `SPEAKER_XX` ids and sample 3–5 segments per speaker

+ 2. Gather context (inline hints, meta.md, transcript clues)

+ 3. Map each `SPEAKER_XX` → real name with a confidence note. If unsure, keep `SPEAKER_XX` rather than guessing

+ 4. Rewrite the `speaker` field across all segments; do not touch `text`, `start`, `end`, or segment order

+ 5. Show the user the mapping before finalizing if any mapping is uncertain

- The `context` parameter accepts a `.meta.md` file with YAML frontmatter:

+ **meta.md** (optional, strong signal):

```yaml

---

title: "Deep Dive into LLMs"

- channel: "Tech Podcast"

speakers:

- name: Alice Chen

role: host

- name: Bob Smith

role: guest

---

- Discussion about large language models, training techniques,

- and the future of AI.

```

- The diarizer uses this to:

- - Match detected speakers to known names

- - Use title/description as context for name inference

- - Preserve speaker roles in output

- ## Output Contract

+ ### CLI-LLM fallback (headless / automated runs)

- Output JSON preserves all source fields plus adds `speaker`:

+ When the agent is not in the loop (batch pipelines, CI, unattended scripts), let the CLI do name inference with its own LLM backend:

- ```json

- {

- "supervisions": [

- {

- "text": "Welcome to the show.",

- "start": 0.0,

- "end": 2.5,

- "speaker": "Alice Chen"

- },

- {

- "text": "Thanks for having me.",

- "start": 3.0,

- "end": 5.1,

- "speaker": "Bob Smith"

- }

- ]

- }

+ ```bash

+ lai config set diarization.llm.model_name gemini-3-flash-preview # one-time

+ # Gemini key: see /lai-transcribe

+ lai diarize run --direct -Y \

+ podcast.mp3 podcast.aligned.json podcast.diarized.json \

+ diarization.infer_speakers=true \

+ diarization.llm.reasoning=true

```

- Speaker labels follow these conventions:

- - With name inference: actual names (`Alice Chen`, `Bob Smith`)

- - Without inference: generic labels (`SPEAKER_00`, `SPEAKER_01`)

- - With context: matched to provided names where possible

- ### Speaker Label Preservation

- If input captions already have speaker labels (from transcription), the

- diarizer recognizes and preserves common formats:

- - `[Alice]` prefix

- - `>> Bob:` prefix

- - `SPEAKER_01:` prefix

- Matched speakers keep their original labels; new speakers get assigned labels.

- ## Workflow

- ### Step 1 -- Verify inputs

- Confirm user has:

- 1. Audio/video file

- 2. Aligned caption file (from `/lai-align`)

- If no aligned captions, guide user to run `/lai-align` first.

- ### Step 2 -- Gather context (optional but recommended)

- Ask user:

- - How many speakers? (if known)

- - Speaker names/roles? (if known)

- - Is there a meta.md file?

- Better context = better speaker identification.

+ - `diarization.infer_speakers=true` — enable CLI-side name inference (requires LLM config above)

+ - `diarization.llm.reasoning=true` — ask the LLM to show its reasoning before committing to a name; trades latency for accuracy on ambiguous speakers

- ### Step 3 -- Configure LLM (if name inference needed)

+ You can also pass hints at invocation time without any LLM:

```bash

- lai config get diarization.llm.model_name

+ lai diarize run podcast.mp3 podcast.aligned.json podcast.diarized.json \

+ context="Host: Alice Chen (tech journalist), Guest: Bob Smith (AI researcher)"

+ # or point at a meta.md (first positional `context` arg also accepts a file path):

+ lai diarize run podcast.mp3 podcast.aligned.json podcast.diarized.json context=podcast.meta.md

```

- If not set and user wants name inference:

- ```bash

- lai config set diarization.llm.model_name gemini-2.5-flash

- lai config set GEMINI_API_KEY your-key

- ```

+ ## Output

- ### Step 4 -- Run diarization

+ Each supervision gains a `speaker` field:

- ```bash

- lai diarize run <audio> <aligned_caption> <output> [options]

+ ```json

+ { "text": "Welcome to the show.", "start": 0.0, "end": 2.5, "speaker": "Alice Chen" }

```

- ### Step 5 -- Review speaker assignments

- Check the output for correct speaker attribution. Common issues:

- - Speakers confused in overlapping speech

- - Short utterances misattributed

- - Speaker count wrong (too many splits or too few)

- ## Error Handling

+ ## Common Issues

- | Condition | Action |

- |-----------|--------|

- | No aligned segments | Input must have timing. Run `/lai-align` first |

- | LLM not configured for inference | Guide user to set `diarization.llm.model_name` |

- | Too many speakers detected | Set `diarization.max_speakers=N` |

- | Speaker names not inferred | Provide context via `context=` parameter |

- | Audio format error | Convert to WAV/MP3/M4A first |

+ | Problem | Fix |

+ |---------|-----|

+ | `No aligned segments` | Run `/lai-align` first |

+ | Too many speakers detected | `diarization.max_speakers=N` |

+ | Agent can't confidently name a speaker | Keep `SPEAKER_XX` and ask the user — don't guess |

+ | Headless run, no LLM configured | `lai config set diarization.llm.model_name gemini-3-flash-preview` |

## Related Skills

- | Skill | Use When |

- |-------|----------|

- | `/lai-align` | Need aligned captions before diarization |

- | `/lai-transcribe` | Need transcript from scratch |

- | `/lai-translate` | Translate diarized captions |

- | `/lai-summarize` | Summarize with speaker attribution |

- | `/lai-youtube` | Full YouTube pipeline |

- ### Common Workflow Chains

- ```bash

- # Full pipeline: Transcribe -> Align -> Diarize

- lai transcribe run podcast.mp3 transcript.json

- lai alignment align podcast.mp3 transcript.json aligned.json

- lai diarize run podcast.mp3 aligned.json diarized.json \

- diarization.infer_speakers=true

- # YouTube -> Align -> Diarize

- lai youtube align "https://youtu.be/xxx" -o aligned.json

- lai diarize run video.mp4 aligned.json diarized.json \

- context=video.meta.md

- ```

- ## Non-Goals

- - Does NOT transcribe audio -- use `/lai-transcribe`

- - Does NOT align captions -- use `/lai-align`

- - Does NOT perform voice cloning or speaker verification

- - Does NOT work on unaligned (no timing) text

- - Does NOT guarantee 100% accuracy on overlapping speech

+ - `/lai-align` — produce the aligned input (required)

+ - `/lai-transcribe` — transcript from scratch

+ - `/lai-translate`, `/lai-summarize` — run on diarized output for speaker-aware results