speech-generation · diff
git:20260313.0dde941 to git:20260319.fc35d3e
150 added, 105 removed. Audit A to A.
---
name: speech-generation
- description: Generate text-to-speech audio using the media-mcp server's generate_speech tool. Use when the user asks to create voiceovers, narration, audio from text, spoken dialogue, podcast-style audio, audiobook readings, announcements, or voice content. Supports 8 voice options, multi-speaker dialogue, and natural language style instructions for controlling tone, pace, and emotion.
+ description: Generate text-to-speech audio using the ElevenLabs MCP server's text_to_speech tool. Use when the user asks to create voiceovers, narration, audio from text, spoken dialogue, podcast-style audio, audiobook readings, announcements, or voice content. Supports voice selection by name or ID, multiple models (multilingual, flash, turbo), stability/similarity/style controls, speed adjustment, and multiple output formats.
---
# Speech Generation (Text-to-Speech)
- Use the `mcp__media-mcp__generate_speech` tool to convert text to natural-sounding speech via Google Gemini.
+ Use the `mcp__ElevenLabs__text_to_speech` tool to convert text to natural-sounding speech via ElevenLabs.
+ ## Quick Reference
+
+ | I want to create... | Read This | Preset to Start With |
+ |---------------------|-----------|---------------------|
+ | Product demo / explainer video | [voiceover.md](references/voiceover.md) | Product Demo or Explainer Video |
+ | Marketing / promo voiceover | [voiceover.md](references/voiceover.md) | Marketing / Promo |
+ | Audiobook / fiction | [narration.md](references/narration.md) | Fiction / Audiobook |
+ | Documentary narration | [narration.md](references/narration.md) | Documentary |
+ | Technical tutorial audio | [documentation.md](references/documentation.md) | Technical Tutorial |
+ | README / docs narration | [documentation.md](references/documentation.md) | README Narration |
+ | Podcast episode | [podcast.md](references/podcast.md) | Conversational Host |
+ | IVR / phone menu | [announcement.md](references/announcement.md) | IVR / Phone Menu |
+ | App notification sound | [announcement.md](references/announcement.md) | App Notification |
+ | Multi-language content | [multilingual.md](references/multilingual.md) | — |
+ | Custom parameter tuning | [voice-settings.md](references/voice-settings.md) | See Named Presets table |
+
## When to Use
- User asks to "read this aloud", "create a voiceover", "generate narration"
- User wants audio versions of text content
- - User needs dialogue with multiple speakers
- - User wants podcast-style audio, announcements, or voice prompts
+ - User needs podcast-style audio, announcements, or voice prompts
- User is creating audio for a video or presentation
## Tool Reference
- ### generate_speech
+ ### text_to_speech
**Key parameters:**
- | Parameter | Type | Description |
- |---|---|---|
- | `text` | string (required) | The text to convert to speech |
- | `voice` | string | Voice selection (8 options available) |
- | `multi_speaker` | object | Speaker-to-voice mapping for dialogue |
- | `style` | string | Natural language style instructions |
+ | Parameter | Type | Default | Description |
+ |---|---|---|---|
+ | `text` | string (required) | — | The text to convert to speech |
+ | `voice_name` | string | — | Name of the voice (e.g. "Rachel", "Adam", "Bella") |
+ | `voice_id` | string | — | Voice ID (alternative to voice_name) |
+ | `model_id` | string | `eleven_multilingual_v2` | Model to use (see Models section) |
+ | `stability` | float (0–1) | 0.5 | Higher = more consistent, lower = more expressive |
+ | `similarity_boost` | float (0–1) | 0.75 | How closely to match the original voice |
+ | `style` | float (0–1) | 0 | Style exaggeration (increases latency if > 0) |
+ | `speed` | float (0.7–1.2) | 1.0 | Speech speed |
+ | `use_speaker_boost` | bool | true | Boost similarity to original speaker |
+ | `language` | string | "en" | ISO 639-1 language code |
+ | `output_format` | string | "mp3_44100_128" | Audio format (see Output Formats) |
+ | `output_directory` | string | ~/Desktop | Where to save the file |
+ **Only one of `voice_name` or `voice_id` can be provided.** If neither is given, the default voice is used.
+
+ ## Models
+
+ | Model ID | Languages | Quality | When to Use |
+ |---|---|---------|-------------|
+ | `eleven_v3` | 70+ | Highest | **Best choice** — newest model, widest language support |
+ | `eleven_multilingual_v2` | 29 | High | Proven fallback if v3 produces artifacts |
+ | `eleven_flash_v2_5` | 32 | Good | Real-time / streaming, ultra-low latency |
+ | `eleven_turbo_v2_5` | 32 | Good | Batch processing, balanced speed/quality |
+ | `eleven_flash_v2` | 1 (EN) | Good | English-only, ultra-low latency |
+ | `eleven_turbo_v2` | 1 (EN) | Good | English-only, balanced |
+ | `eleven_monolingual_v1` | 1 (EN) | Basic | Legacy — avoid unless reproducing old output |
+
+ For detailed model comparison and selection guidance, see [voice-settings.md](references/voice-settings.md).
+
## Voice Selection
- Choose a voice that matches the content's tone and audience. The tool provides 8 distinct voices — experiment to find the best fit for each use case.
+ Use `mcp__ElevenLabs__search_voices` to find voices already in the user's library, or `mcp__ElevenLabs__search_voice_library` to browse the full ElevenLabs voice library.
### Matching voices to content
| Content type | Voice criteria |
|---|---|
| Technical tutorial | Clear, measured, neutral pace |
| Marketing/promo | Energetic, warm, confident |
| Narration/story | Expressive, varied pacing |
| Announcement | Authoritative, clear, professional |
| Conversational | Natural, relaxed, friendly |
- ## Style Instructions
-
- The `style` parameter accepts natural language descriptions of how the speech should sound.
-
- ### Tone
- - "Speak in a warm, friendly tone"
- - "Professional and authoritative"
- - "Casual and conversational, like talking to a friend"
- - "Calm and soothing, like a meditation guide"
- - "Excited and enthusiastic"
-
- ### Pace
- - "Speak slowly and clearly, with pauses between sentences"
- - "Normal conversational pace"
- - "Quick and energetic"
- - "Measured and deliberate, emphasizing key words"
-
- ### Emotion
- - "Speak with genuine curiosity and wonder"
- - "Confident and reassuring"
- - "Urgent, like breaking news"
- - "Gentle and empathetic"
-
- ### Combined style examples
- ```
- "Speak in a warm, professional tone at a moderate pace.
- Emphasize technical terms slightly. Add brief pauses
- before important points."
- ```
+ ### Voice design
- ```
- "Casual and upbeat, like a tech YouTuber explaining
- a cool new feature. Slightly faster pace, with
- enthusiasm on key benefits."
- ```
+ Use `mcp__ElevenLabs__text_to_voice` to generate a custom voice from a text description (e.g. "A warm female voice with a slight British accent"). This creates 3 preview variations. Save the best one with `mcp__ElevenLabs__create_voice_from_preview`.
- ## Multi-Speaker Dialogue
+ ### Voice cloning
- For content with multiple speakers, use the `multi_speaker` parameter to assign voices.
+ Use `mcp__ElevenLabs__voice_clone` to create an instant voice clone from audio files.
- ### Format
+ ## Voice Controls
- Mark speakers in the text, then map them to voices:
+ Controls are summarized below. For parameter interaction details, named presets, and advanced tuning, see [voice-settings.md](references/voice-settings.md).
- **Text:**
- ```
- [Host]: Welcome to the show! Today we're talking about AI.
- [Guest]: Thanks for having me. I'm excited to dive in.
- [Host]: Let's start with the basics. What is machine learning?
- [Guest]: At its core, machine learning is about pattern recognition...
- ```
+ | Parameter | Low | Medium | High |
+ |-----------|-----|--------|------|
+ | **Stability** (0–1) | Expressive, emotional (0.1–0.3) | Balanced default (0.4–0.6) | Consistent, monotone (0.7–1.0) |
+ | **Similarity Boost** (0–1) | More variation from base voice | — | Closely matches original (0.7–1.0) |
+ | **Style** (0–1) | Fastest generation (0) | Moderate expression (0.2–0.3) | Maximum style, higher latency (0.5+) |
+ | **Speed** (0.7–1.2) | Slow, deliberate (0.7) | Normal (1.0) | Fast-paced (1.2) |
- **multi_speaker mapping:**
- ```json
- {
- "Host": "voice_1",
- "Guest": "voice_2"
- }
- ```
+ ## Output Formats
- ### Use cases for multi-speaker
+ | Format | Description |
+ |---|---|
+ | `mp3_44100_128` | MP3 128kbps (default, good balance) |
+ | `mp3_44100_192` | MP3 192kbps (higher quality, Creator tier+) |
+ | `pcm_16000` | PCM 16kHz (raw audio) |
+ | `pcm_44100` | PCM 44.1kHz (high quality raw, Pro tier+) |
+ | `opus_48000_128` | Opus 128kbps (efficient streaming) |
+ | `ulaw_8000` | μ-law 8kHz (Twilio compatible) |
- - **Podcast simulation**: Host + guest conversation
- - **Tutorial dialogue**: Instructor + student Q&A
- - **Product demo**: Narrator + user voices
- - **Audiobook**: Different characters with distinct voices
- - **Interview**: Interviewer + interviewee
+ For platform-specific format recommendations, see [announcement.md](references/announcement.md).
## Common Patterns
### Documentation narration
- Convert a README or guide into audio:
+ See [documentation.md](references/documentation.md) for text cleaning, code block handling, and segmentation.
```
- Text: [the documentation content]
- Style: "Clear and professional, like a technical narrator.
- Moderate pace with pauses between sections.
- Pronounce technical terms carefully."
+ text: [cleaned documentation content]
+ voice_name: "Rachel"
+ model_id: "eleven_v3"
+ stability: 0.55
+ similarity_boost: 0.80
+ style: 0.10
+ speed: 0.90
```
### Video voiceover
- Create narration for a product video:
+ See [voiceover.md](references/voiceover.md) for timing tables, text prep, ffmpeg mixing, the timed voiceover pipeline, and golden rules.
```
- Text: "Introducing our new dashboard. With real-time analytics,
- you can track performance at a glance. Drag and drop widgets
- to customize your view. Export reports with a single click."
- Style: "Confident and polished, like a product launch video.
- Measured pace, slight emphasis on feature names."
+ text: "Introducing our new dashboard. With real-time analytics,
+ you can track performance at a glance."
+ voice_name: "Adam"
+ model_id: "eleven_v3"
+ stability: 0.50
+ similarity_boost: 0.80
+ style: 0.25
+ speed: 0.95
```
### Announcement / notification
- Short-form audio for apps or systems:
+ See [announcement.md](references/announcement.md) for brevity rules, templates, and platform formats.
```
- Text: "Your deployment is complete. All 12 services are running."
- Style: "Brief and clear, neutral professional tone."
+ text: "Deployment complete. All 12 services are running."
+ voice_name: "Bella"
+ model_id: "eleven_v3"
+ stability: 0.75
+ similarity_boost: 0.85
+ style: 0.05
+ speed: 1.05
```
### Story / narrative
- Creative content narration:
+ See [narration.md](references/narration.md) for character voices, long-form strategy, and pacing.
```
- Text: [story content]
- Style: "Expressive storyteller voice, varied pacing — slower
- for dramatic moments, slightly faster for action.
- Pause before dialogue."
+ text: [story content]
+ voice_name: "Rachel"
+ model_id: "eleven_v3"
+ stability: 0.25
+ similarity_boost: 0.80
+ style: 0.50
+ speed: 0.90
```
- ### Multi-language consideration
- The tool generates speech based on the text language. Write text in the target language for non-English output.
+ ### Multi-language
+ See [multilingual.md](references/multilingual.md) for language tiers, pronunciation hints, and batch workflows.
- ## Output Handling — MEDIA_OUTPUT_DIR
+ Set `language` to the ISO 639-1 code and write text in the target language. Use `eleven_v3` (70+ languages) or `eleven_multilingual_v2` (29 languages).
- **When `MEDIA_OUTPUT_DIR` is set** (recommended): The MCP server saves the generated speech audio to a file and returns only the file path. Always use just this path — do NOT request or embed the raw audio data. This is critical because all MCP request/response messages are stored in the conversation history, and large base64 payloads pollute the context window, degrading performance.
+ ### Podcast / multi-speaker
+ See [podcast.md](references/podcast.md) for multi-speaker workflow, voice pairing, and episode structure.
- **When `MEDIA_OUTPUT_DIR` is not set**: The MCP server has no choice but to return the audio as base64 data in the response. This works but is suboptimal for conversation history size.
+ Generate each speaker's lines separately with different voices, then concatenate with natural gaps.
+ ## Additional ElevenLabs Tools
+
+ | Tool | Use case |
+ |---|---|
+ | `mcp__ElevenLabs__speech_to_speech` | Transform audio from one voice to another |
+ | `mcp__ElevenLabs__speech_to_text` | Transcribe audio to text (with optional diarization) |
+ | `mcp__ElevenLabs__isolate_audio` | Isolate vocals from background noise |
+ | `mcp__ElevenLabs__voice_clone` | Clone a voice from audio files |
+ | `mcp__ElevenLabs__text_to_voice` | Design a new voice from a text description |
+
## Combining with Other Media Skills
### Voiceover + Video
1. Generate the video with `generate_video`
- 2. Generate the voiceover with `generate_speech`
- 3. Combine with ffmpeg: `ffmpeg -i video.mp4 -i voiceover.wav -c:v copy -c:a aac output.mp4`
+ 2. Generate the voiceover with `mcp__ElevenLabs__text_to_speech`
+ 3. Combine with ffmpeg: `ffmpeg -i video.mp4 -i voiceover.mp3 -c:v copy -c:a aac output.mp4`
### Narration + Background Music
- 1. Generate narration with `generate_speech`
+ 1. Generate narration with `mcp__ElevenLabs__text_to_speech`
2. Generate background music with `generate_music`
- 3. Mix audio: `ffmpeg -i narration.wav -i music.wav -filter_complex "[1:a]volume=0.2[bg];[0:a][bg]amix=inputs=2:duration=longest" output.wav`
+ 3. Mix audio: `ffmpeg -i narration.mp3 -i music.wav -filter_complex "[1:a]volume=0.2[bg];[0:a][bg]amix=inputs=2:duration=longest" output.mp3`
### Podcast Production
- 1. Write dialogue with speaker markers
- 2. Generate with multi-speaker mode
- 3. Generate intro/outro jingle with `generate_music`
- 4. Concatenate: `ffmpeg -i intro.wav -i dialogue.wav -i outro.wav -filter_complex "concat=n=3:v=0:a=1" podcast.wav`
+ 1. Generate each speaker's lines separately with different voices
+ 2. Generate intro/outro jingle with `generate_music`
+ 3. Concatenate: `ffmpeg -i intro.wav -i part1.mp3 -i part2.mp3 -i outro.wav -filter_complex "concat=n=4:v=0:a=1" podcast.mp3`
+ See [podcast.md](references/podcast.md) for the complete step-by-step workflow.
+
## Tips
- - Write text as you want it spoken — use punctuation for natural pauses (commas, periods, ellipses)
+ - Write text as you want it spoken — use punctuation for natural pauses
- Use "..." for longer pauses: "And the winner is... congratulations!"
- Spell out abbreviations if you want them read as words: "API" vs "A.P.I."
- For numbers, write them as words if pronunciation matters: "twenty-three" vs "23"
- - Test with a short sentence first to verify the voice and style before generating long content
- - Multi-speaker mode produces more natural results for dialogue than generating each speaker separately
+ - Test with a short sentence first to verify the voice before generating long content
+ - Use `output_directory` to control where files are saved (defaults to ~/Desktop)
+ - For advanced parameter tuning, consult [voice-settings.md](references/voice-settings.md)
+ - For video voiceover, follow the [Golden Rules](references/voiceover.md#golden-rules-for-video-voiceover) and [Timed Voiceover Pipeline](references/voiceover.md#timed-voiceover-pipeline)
+ - Check [voice tiers](references/voice-settings.md#voice-tiers--api-access) before selecting a voice — library/cloned voices need a paid plan
+ - Use the [Voice Selection Guide](references/voiceover.md#voice-selection-guide) to match voice type to content — always test the voice before writing the full script
+
+ ## Reference Files
+
+ | File | What It Covers | When to Read |
+ |------|---------------|--------------|
+ | [voice-settings.md](references/voice-settings.md) | Parameter interactions, named presets, model selection, output formats | Tuning voice parameters or choosing a model |
+ | [voiceover.md](references/voiceover.md) | Product demos, explainers, marketing, corporate training | Creating audio for video content |
+ | [narration.md](references/narration.md) | Audiobooks, fiction, documentaries, children's stories | Long-form narrative or character voices |
+ | [documentation.md](references/documentation.md) | Tutorials, READMEs, API docs, e-learning | Converting technical docs to audio |
+ | [podcast.md](references/podcast.md) | Multi-speaker, interviews, episode structure | Podcast or dialogue production |
+ | [announcement.md](references/announcement.md) | IVR, notifications, system alerts, public address | Short-form announcements or telephony |
+ | [multilingual.md](references/multilingual.md) | Language support, pronunciation, localization | Non-English or multi-language content |