git:20260417.89d056b to git:20260428.4ab9c84

- description: Use when user needs accurate/precise caption timing, or aligning captions with audio/video using forced alignment. Corrects caption timing to match actual speech. Uses LattifAI Lattice-1 model. Use this skill whenever the user mentions subtitle timing, caption sync, audio-text alignment, word-level timestamps, karaoke timing, or has both an audio file and a text file that need to be synchronized. Also triggers on "align captions", "fix timing", "forced alignment", "对齐字幕", "校准时间轴", "时间轴不准", "字幕同步", "timestamps are off", "captions are out of sync", even if they don't use the word "alignment" explicitly.

- Precision audio-text forced alignment powered by the Lattice-1 model. Produces

- word-level and segment-level timestamps with sub-frame accuracy.

-

- **Requires LattifAI API Key** -- get one free via `lai auth trial` (see `/lai-setup`).

-

- **CLI syntax note**: LattifAI uses `nemo_run` configuration syntax. Options use

- `key=value` format (e.g., `media.streaming_chunk_secs=300`), NOT traditional

- flags (`--streaming-chunk-secs`). This applies to all `lai` subcommands.

-

- ## When to Use

-

- - **Precise timing needed** -- user requests accurate timestamps or precise alignment

- - **Sync misaligned captions** -- fix timing drift in downloaded or auto-generated captions

- - **Align manual transcripts** -- match hand-written text to speech precisely

- - **Post-transcription alignment** -- improve coarse timestamps from ASR output

- - **Word-level timestamps** -- need per-word timing for karaoke, highlighting, or TTS

- - **Multi-format input** -- SRT, VTT, ASS, TXT, MD, JSON, TextGrid, TSV, and 20+ more

-

- ## When NOT to Use

- - No existing text/transcript -- use `/lai-transcribe` first to generate one

- - Only need format conversion -- use `/lai-caption` instead

- - Need speaker identification -- use `/lai-diarize` after alignment

- - Very short clips (<3 seconds) -- alignment needs sufficient audio context

- ## Prerequisites

- lai doctor # Verify environment

- lai auth whoami # Verify API key

- If not set up, run `/lai-setup` first.

-

- ## CLI Usage

-

- ### Basic Alignment

- ```bash

- lai alignment align audio.wav caption.srt output.srt

- ```

- Shortcut form:

- ```bash

- lai-align audio.wav caption.srt output.srt

- ```

- ### Word-Level Timestamps

- ```bash

- lai alignment align audio.mp3 caption.srt output.json \

- caption.render.word_level=true

- ```

- Word-level output is best in JSON format (preserves per-word timing data):

- "supervisions": [

- {

- "text": "Hello world",

- "start": 0.0,

- "end": 2.5,

- "alignment": [

- {"word": "Hello", "start": 0.0, "end": 0.5},

- {"word": "world", "start": 0.6, "end": 2.5}

- ]

- }

- ]

- ### Smart Sentence Segmentation

-

- For word-level or poorly segmented input (e.g., YouTube VTT):

-

- ```bash

- lai alignment align video.mp4 caption.vtt output.srt \

- caption.input.split_sentence=true

- ```

-

- Uses `wtpsplit` AI model to re-segment into natural sentences. Especially

- useful for YouTube auto-captions where segments break mid-sentence.

-

- ### Long Audio (Streaming Mode)

-

- For audio longer than 10 minutes, enable streaming to limit memory usage:

-

- ```bash

- lai alignment align long_podcast.mp3 transcript.srt output.srt \

- media.streaming_chunk_secs=300

- ```

-

- Supports audio up to 20 hours. Default chunk: 300 seconds (5 minutes).

-

- ## Parameters

-

- | Parameter | CLI Syntax | Default | Description |

- |-----------|-----------|---------|-------------|

- | Input media | positional arg 1 | required | Audio or video file path |

- | Input caption | positional arg 2 | required | Caption/transcript file path |

- | Output caption | positional arg 3 | required | Output file path (format from extension) |

- | Word-level | `caption.render.word_level=true` | `false` | Include per-word timestamps |

- | Split sentence | `caption.input.split_sentence=true` | `false` | AI sentence re-segmentation |

- | Streaming | `media.streaming_chunk_secs=N` | off | Chunk size for long audio (seconds) |

- | Include speaker | `caption.render.include_speaker_in_text=true` | `false` | Embed speaker labels in text |

- | Normalize text | `caption.input.normalize_text=true` | `false` | Normalize input text before alignment |

- | API key | `client.api_key=xxx` | from config | Override API key |

- | Device | `alignment.device=cuda` | auto | Force device (cuda/mps/cpu) |

-

- ## Supported Formats

-

- **Input** (auto-detected from extension/content):

-

- | Format | Extensions |

- |--------|-----------|

- | SubRip | `.srt` |

- | WebVTT | `.vtt` |

- | ASS/SSA | `.ass`, `.ssa` |

- | Plain text | `.txt` |

- | Markdown (Gemini) | `.md` |

- | JSON | `.json` |

- | TextGrid (Praat) | `.TextGrid` |

- | TSV/CSV | `.tsv`, `.csv` |

- | Audacity | `.aud` |

- | LRC (lyrics) | `.lrc` |

- | TTML | `.ttml`, `.xml` |

- | And 15+ more | See `lai alignment align --help` |

-

- **Output**: Same formats as input. Format is inferred from output file extension.

-

- **Recommended output format**: JSON (preserves word-level timing for downstream use).

- Convert to other formats later with `/lai-caption`.

-

- ## Workflow

-

- ### Step 1 -- Verify setup

-

- ```bash

- lai auth whoami

- ```

-

- If not authenticated, guide user through `/lai-setup`.

-

- ### Step 2 -- Identify inputs

-

- Determine:

- 1. **Audio/video file** -- the media source

- 2. **Caption/transcript file** -- the text to align

- 3. **Desired output format** -- SRT for playback, JSON for downstream processing

-

- ### Step 3 -- Decide options

-

- | Scenario | Recommended Options |

- |----------|-------------------|

- | YouTube auto-captions (word-level VTT) | `caption.input.split_sentence=true` |

- | Long podcast (>10 min) | `media.streaming_chunk_secs=300` |

- | Need per-word timing | `caption.render.word_level=true`, output as `.json` |

- | Multi-speaker content | Align first, then `/lai-diarize` |

- | Want ASS with styling | Output as `.json` first, then `/lai-caption` to convert |

-

- ### Step 4 -- Run alignment

-

- ```bash

- lai alignment align <audio> <caption> <output> [options]

- ```

-

- ### Step 5 -- Verify output

-

- Read the first few segments of the output to confirm timing looks correct.

- For JSON output, check that `start`/`end` values are reasonable.

-

- ## Output Contract

-

- The output preserves all input text content. Only timestamps change.

-

- - **Segment-level**: Each caption segment gets precise `start` and `end` timestamps

- - **Word-level** (when enabled): Each word within a segment gets its own `start`/`end`

- - **Speaker labels**: Preserved from input if present

- - **Non-speech events**: `[MUSIC]`, `[APPLAUSE]` etc. are preserved with timing

- ## Error Handling

- | Condition | Action |

- |-----------|--------|

- | `API KEY verification error` | Key invalid/expired. Run `lai auth whoami` to check, then `lai auth login` or `lai auth trial` |

- | `No segments found` | Input caption is empty or unreadable. Check file format |

- | `Audio format error` | Convert to WAV/MP3/M4A first: `ffmpeg -i input.xxx -ar 16000 output.wav` |

- | `CUDA out of memory` | Add `alignment.device=cpu` or reduce `media.streaming_chunk_secs` |

- | `Model download failed` | Check network. Model auto-downloads from HuggingFace on first run |

- | Timing looks wrong | Try `caption.input.normalize_text=true` or `caption.input.split_sentence=true` |

- | Skill | Use When |

- |-------|----------|

- | `/lai-setup` | Need to install or authenticate first |

- | `/lai-transcribe` | No transcript exists yet -- generate one |

- | `/lai-diarize` | Need speaker labels after alignment |

- | `/lai-caption` | Convert aligned output to another format |

- | `/lai-youtube` | Working with YouTube videos (download + align combined) |

- | `/lai-translate` | Translate aligned captions to another language |

-

- ### Common Workflow Chains

-

- ```bash

- # Transcribe then align (most common)

- lai transcribe run video.mp4 transcript.json

- lai alignment align video.mp4 transcript.json aligned.json

-

- # Align then convert to multiple formats

- lai alignment align audio.mp3 caption.srt aligned.json caption.render.word_level=true

- laicap-convert aligned.json aligned.srt

- laicap-convert aligned.json aligned.vtt

- laicap-convert aligned.json aligned.ass

-

- # Align then diarize

- lai alignment align podcast.mp3 transcript.srt aligned.json

- lai diarize run podcast.mp3 aligned.json diarized.json

- ```

-

- ## Non-Goals

-

- - Does NOT generate transcripts -- use `/lai-transcribe` for that

- - Does NOT identify speakers -- use `/lai-diarize` after alignment

- - Does NOT translate text -- use `/lai-translate` for translation

- - Does NOT download media -- use `/lai-youtube` or `/lai-caption` for downloads

- - Does NOT modify caption text content -- only timestamps change