flutter-edge-ai-speech git:20261030.22d611aA

Immutable. This exact content never changes and is served at /api/v1/blob/b73663d3a277edf9.

---
name: flutter-edge-ai-speech
description: Use when adding speech to a flutter_edge_ai app — speech-to-text (transcribe a voice note, dictation, Whisper, moonshine, Parakeet), text-to-speech (Matcha, Qwen3-TTS, Inflect), or a push-to-talk voice assistant with VoiceSession. Also use when transcripts come back in English for non-English audio, a WAV file has to become 16 kHz PCM, synthesized audio plays at the wrong pitch, or getActiveTts throws a StateError about the language. For audio sent straight to Gemma in a chat, use flutter-edge-ai-inference.
---

# Speech with flutter_edge_ai_speech

## Rules

1. Depend on `flutter_edge_ai` and `flutter_edge_ai_speech`, and import both. The speech package does not re-export core.
2. `transcribe` takes raw PCM — 16 kHz, mono, 16-bit little-endian, as a `Uint8List` — and returns the text. Not a WAV file, not 44.1 or 48 kHz: nothing resamples or converts it.
3. Play synthesized audio at `synth.sampleRate`. It differs per model.
4. Only Whisper has a selectable output language. moonshine-tiny and Parakeet are English-only, and passing a language to them throws `ArgumentError`.
5. STT language: set a default with `getActiveStt(language:)` or override one call with `transcribe(pcm, language:)`. Nothing reloads.
6. TTS language: `close()` the synthesizer first. Asking a live synthesizer for another language throws `StateError`.
7. Android needs `minSdk 30`. There is no web support — the web backends throw `UnsupportedError`.
8. Close recognizers and synthesizers.

## Setup

```sh
flutter pub add flutter_edge_ai flutter_edge_ai_speech
```

```dart
import 'package:flutter_edge_ai/flutter_edge_ai.dart';
import 'package:flutter_edge_ai_speech/flutter_edge_ai_speech.dart';

await FlutterEdgeAi.initialize(
  sttBackends: [LiteRtSttBackend()],
  ttsBackends: [LiteRtTtsBackend()],
);
```

Speech runs on the same native libraries as the `.litertlm` engine. Its build setup — Android `minSdk 30`, the Apple entries — is in the flutter-edge-ai-inference skill's [platform setup](../flutter-edge-ai-inference/references/platform-setup.md), installed alongside this one.

## Speech-to-text

An STT model is two files — the model and its tokenizer — usually from different repos. `install()` skips files already on disk.

```dart
await FlutterEdgeAi.installStt()
    .modelFromNetwork('https://huggingface.co/litert-community/whisper-tiny/resolve/main/whisper_tiny_30s_f32.tflite')
    .tokenizerFromNetwork('https://huggingface.co/openai/whisper-tiny/resolve/main/tokenizer.json')
    .ofType(SttModelType.whisper)
    .install();

final SpeechRecognizer recognizer = await FlutterEdgeAi.getActiveStt(language: 'de');
try {
  final String german = await recognizer.transcribe(germanPcm);
  final String french = await recognizer.transcribe(frenchPcm, language: 'fr');
} finally {
  await recognizer.close();
}
```

| `SttModelType` | Languages | Window |
| --- | --- | --- |
| `SttModelType.moonshine` | English | 5 s |
| `SttModelType.whisper` | 99, selectable, default `'en'` | 30 s |
| `SttModelType.parakeet` | English | 5 s; f32 is 2.35 GB, for desktop — on a Galaxy S23 (8 GB) its GPU load was killed. On a phone use the int8 `parakeet_ctc_0.6b_5s_i8.tflite` (596 MB) on the CPU (#747) |

On the GPU, run the f32 files: on Adreno 740 the int8 (`_i8`) ones did not compile there, and the load fails with an error naming the file.

**Audio longer than the window is silently truncated**, not rejected: it is zero-padded when shorter and cut when longer, so a 40-second clip on Whisper returns the first 30 seconds with no error. Split long recordings yourself.

Whisper tiny is weak outside English. Whisper base int8 is the next size up in the catalog; install it the same way from `https://huggingface.co/litert-community/whisper-base/resolve/main/whisper_base_30s_i8.tflite` with the tokenizer `https://huggingface.co/openai/whisper-base/resolve/main/tokenizer.json`.

## Getting 16 kHz mono PCM

The package has no resampler and no WAV reader. Record in the right format from the start — with the `record` package:

```dart
import 'package:record/record.dart';

const config = RecordConfig(
  encoder: AudioEncoder.wav,
  sampleRate: 16000,
  numChannels: 1,
);
```

Recording needs microphone access. On Android `record` declares `android.permission.RECORD_AUDIO` in its own manifest and the manifest merger adds it to the app; iOS and macOS need entries of your own:

```xml
<!-- ios/Runner/Info.plist and macos/Runner/Info.plist -->
<key>NSMicrophoneUsageDescription</key>
<string>Speech is transcribed on this device.</string>

<!-- macos/Runner/DebugProfile.entitlements and macos/Runner/Release.entitlements -->
<key>com.apple.security.device.audio-input</key>
<true/>
```

The macOS sandbox withholds the microphone without the entitlement, and it has to be in both entitlements files. On Android, iOS and macOS the user must also allow it at run time; `hasPermission()` asks, so call it before recording:

```dart
final recorder = AudioRecorder();
if (!await recorder.hasPermission()) return;
```

That produces a WAV file. Its header is not always 44 bytes — take the samples from the `data` chunk:

```dart
import 'dart:typed_data';

/// The samples of a 16 kHz mono 16-bit WAV file, without its header.
Uint8List pcmFromWav(Uint8List wav) {
  final view = ByteData.sublistView(wav);
  var offset = 12; // after 'RIFF', the size and 'WAVE'
  while (offset + 8 <= wav.length) {
    final id = String.fromCharCodes(wav, offset, offset + 4);
    final size = view.getUint32(offset + 4, Endian.little);
    final start = offset + 8;
    if (id == 'data') {
      final end = start + size > wav.length ? wav.length : start + size;
      return Uint8List.sublistView(wav, start, end);
    }
    offset = start + size + (size & 1); // chunks are padded to an even size
  }
  throw const FormatException('WAV file has no data chunk');
}
```

A file recorded at another rate or channel count — 44.1 kHz stereo, say — has to be converted first: average the channels to mono, then resample with a low-pass filter. Dropping samples instead aliases and costs accuracy.

## Traps

**Transcript comes back in English**
- Symptom: German audio, fluent English text, no error.
- Cause: Whisper's language token decides the output language, not what it understands — with `'en'` it translates. moonshine only ever produces English.
- Fix: use Whisper and pass `language:`.

**A language is rejected**
- Whisper codes are bare and lowercase: `'de'`, not `'de-DE'`, `'DE'` or `'german'`. Malformed codes throw `ArgumentError` from `getActiveStt`.
- A well-formed code the installed checkpoint lacks (e.g. `'zz'`) throws `ArgumentError` from `transcribe`.

## Text-to-speech

```dart
await FlutterEdgeAi.installTts()
    .fromNetwork('https://huggingface.co/litert-community/Matcha-TTS/resolve/main/')
    .ofType(TtsModelType.matcha)
    .install();

final SpeechSynthesizer synth = await FlutterEdgeAi.getActiveTts();
try {
  final audio = await synth.synthesize('Hello world.'); // 16-bit PCM
  final rate = synth.sampleRate;                        // 22050 for Matcha
} finally {
  await synth.close();
}
```

| `TtsModelType` | Languages |
| --- | --- |
| `TtsModelType.matcha` | English, fixed by the installed bundle — it ignores `language:` |
| `TtsModelType.qwen3` | `chinese`, `english`, `german`, `italian`, `portuguese`, `spanish`, `japanese`, `korean`, `french`, `russian`, or `auto` |
| `TtsModelType.inflect` | English |

`TtsModelType.supertonic` and `TtsModelType.kokoro` are in the enum but throw `UnimplementedError` — do not use them.

To ship the voice with the app instead of downloading it, swap `fromNetwork` for one of:

| Call | Expects | Notes |
| --- | --- | --- |
| `fromAsset('assets/tts/matcha/')` | Flutter asset directory laid out like the HF repo | Declare every subdirectory in `pubspec.yaml` (Qwen3 has `tables/` and `voices/`); copied into app storage |
| `fromFile('/abs/dir')` | Directory on disk, same layout | Used in place and stays the app's — `uninstallTts()` leaves these files |
| `fromBundled()` | Native resources named `<type>__<file>`, e.g. `matcha__config.json` | Android `assets/models/`, iOS Runner target; Android and iOS only |

Inflect's four Matcha G2P files go next to its own two. A missing file fails `install()` with the full list before the current voice is replaced; a switch that fails partway (lost connection) leaves no voice active, so `getActiveTts` throws until `install()` is run again. `install()` throws `UnsupportedError` on web. To ship a changed voice in an app update, use a new asset directory (a bundled voice refreshes itself); with `fromFile` on iOS, call `install()` at launch, because the data directory moves on update.

Switching the Qwen3 language — full lowercase names, not ISO codes, and only with the Qwen3 bundle installed (Matcha still throws the same `StateError` but the language changes nothing):

```dart
final english = await FlutterEdgeAi.getActiveTts(language: 'english');
await english.close();
final german = await FlutterEdgeAi.getActiveTts(language: 'german');
```

Without the `close()`, the second call throws `StateError: Active TTS synthesizer was created for language 'english'; call close() before requesting 'german'.`

## Voice assistant

`VoiceSession` runs one push-to-talk turn: transcribe, generate, speak, with barge-in. It uses the recognizer's current language.

Wrap the loop in `try`/`catch`: a failed stage — transcribe, generate or synthesize — arrives as a **stream error**, not as an event. `VoiceErrorEvent` is reserved in this release and never emitted; the `case` is only there because the switch must be exhaustive. To barge in, call `await voice.interrupt()` — cancelling the subscription is not a portable stop.

```dart
final reply = StringBuffer();
final voice = VoiceSession.fromChat(
  recognizer: await FlutterEdgeAi.getActiveStt(language: 'de'),
  chat: chat,
  synthesizer: await FlutterEdgeAi.getActiveTts(),
);

await for (final event in voice.runTurn(pcm16kMono)) {
  switch (event) {
    case VoiceTranscriptEvent(:final text):
      print('heard: $text');
    case VoiceReplyTextEvent(:final chunk):
      reply.write(chunk);
    case VoiceReplyAudioEvent(:final sampleRate):
      print('audio at $sampleRate Hz');
    case VoiceTurnInterruptedEvent():
      print('interrupted — stop the player');
    case VoiceTurnCompleteEvent():
      print('done');
    case VoiceErrorEvent(:final error):
      print('failed: $error');
  }
}
```

`chat` is an `InferenceChat` from the flutter-edge-ai-inference skill. A chat created with tools also needs `onToolCall:` — without it `fromChat` throws.

### Audio turns: no speech-to-text

A model that hears audio (Gemma 4 E2B or E4B as `.litertlm`, native platforms only) can take the recording itself: `runAudioTurn` skips the transcription and sends the PCM to the chat. Set `supportAudio: true` on `getActiveModel`, which loads the audio encoder; `createChat` takes the flag from the model unless you pass it, and on a chat whose `supportAudio` is false `runAudioTurn` throws `StateError`. It takes the same 16 kHz mono 16-bit PCM as `runTurn`, and wraps it in a WAV header, because a `.litertlm` model decodes an audio file; a custom `VoiceResponder.respondToAudio` gets that WAV and can pass it to `AgentSession.ask(audioBytes:)` as is. There is no `VoiceTranscriptEvent`, the complete and interrupted events carry an empty transcript, and a recording under 2 bytes completes without asking the model. After a barge-in the chat keeps earlier audio turns with `flutter_edge_ai_litertlm` 1.15.0 and later; with 1.14.0 and earlier, and on the NPU, it is rebuilt from text and forgets them. Requires `flutter_edge_ai_speech` 0.5.7.

```dart
final model = await FlutterEdgeAi.getActiveModel(
  maxTokens: 4096,
  supportAudio: true,
);
final chat = await model.createChat(supportAudio: true, maxOutputTokens: 128);
final voice = VoiceSession.fromChat(
  recognizer: recognizer,
  chat: chat,
  synthesizer: synth,
);

await for (final event in voice.runAudioTurn(pcm16kMono)) {
  if (event is VoiceReplyTextEvent) print(event.chunk);
}
```