fetcher-llm-streaming · diff
git:20260818.2ff7b22 to git:20260924.b36c6c8
32 added, 15 removed. Audit A to A.
---
name: fetcher-llm-streaming
description: >
- Consume SSE and LLM token streams with `@ahoo-wang/fetcher-eventstream`: Response prototype helpers, standalone converters, termination detectors, result extractors, and ReadableStream async iteration. Use for `eventStream`, `jsonEventStream`, or token-by-token responses.
+ Consume Server-Sent Events and LLM token streams with `@ahoo-wang/fetcher-eventstream`: the `Response.prototype` helpers (`eventStream`, `jsonEventStream`), standalone converters, `TerminateDetector` for `[DONE]`, SSE result extractors and `for await` iteration. Use for a custom SSE endpoint or non-OpenAI token stream. Not for OpenAI/GPT chat completions, even streamed — fetcher-openai-client handles those and their `[DONE]`.
---
# fetcher-llm-streaming
- ## Workflow
+ ## Decisions
- 1. Import `@ahoo-wang/fetcher-eventstream` for side-effect prototype helpers when using `Response` extensions.
- 2. Use standalone conversion functions when prototype mutation is undesirable.
- 3. Handle stream conversion errors explicitly with `EventStreamConvertError`.
- 4. For OpenAI-style DONE termination, supply a detector function (`TerminateDetector`); the ready-made `DoneDetector` lives in `@ahoo-wang/fetcher-openai`.
- 5. Load `references/api.md` for pipeline details, OpenAI streaming examples, and UI update patterns.
+ - **Prototype helpers vs converters**: `import '@ahoo-wang/fetcher-eventstream'` patches `Response.prototype` (`contentType`, `isEventStream`, `eventStream()`, `requiredEventStream()`, `jsonEventStream()`, `requiredJsonEventStream()`, skipping members that already exist) and polyfills `ReadableStream` async iteration. The standalone converters `toServerSentEventStream(response)` and `toJsonServerSentEventStream(stream, detector)` avoid _calling_ the patched members, but importing them (or anything else from the package) still runs the patch — there is no side-effect-free entry. If `Response.prototype` must stay untouched, do not depend on this package.
+ - **Nullable vs required**: `eventStream()` / `jsonEventStream()` return `null` for a non-SSE Content-Type; the `required*` variants throw `EventStreamConvertError` (with `.response`).
+ - **OpenAI chat completions** already handle `[DONE]` and typing — use `$fetcher-openai-client` instead of rebuilding it here.
- ## Key Practices
+ ## Gotchas a capable model gets wrong
- - Keep parsing, termination detection, and UI state updates as separate steps.
- - Use async iteration over streams to avoid buffering full responses in memory.
- - When pairing with decorators, configure result extractors at the endpoint boundary.
+ - There is **no default terminator**. Without a `TerminateDetector`, a `data: [DONE]` line reaches `JSON.parse` and the iteration throws `SyntaxError`. `DoneDetector` lives in `@ahoo-wang/fetcher-openai`, not here.
+ - `JsonEventStreamResultExtractor` passes no detector; for `[DONE]` endpoints write a `ResultExtractor` that calls `exchange.requiredResponse.requiredJsonEventStream(detector)`.
+ - The SSE extractors are standalone exports of this package, not members of `ResultExtractors` from `@ahoo-wang/fetcher`.
+ - Items are `JsonServerSentEvent<T>`: the payload is `event.data`; `id` is `''` when the server sent none and `event` defaults to `'message'`.
+ - Errors during iteration are `SyntaxError` or network/stream errors, not `EventStreamConvertError` — handle both around the `for await` loop.
+ ## Minimal example
+
+ ```ts
+ import '@ahoo-wang/fetcher-eventstream';
+ import type { TerminateDetector } from '@ahoo-wang/fetcher-eventstream';
+ import { fetcher } from './http';
+
+ const done: TerminateDetector = e => e.data === '[DONE]';
+ const response = await fetcher.post('/generate', { body: { prompt } });
+ let text = '';
+ for await (const event of response.requiredJsonEventStream<{ token: string }>(
+ done,
+ )) {
+ text += event.data.token;
+ }
+ ```
+
## References
- - `references/api.md`: Detailed package API, examples, and edge-case guidance. Load it only when the task needs prototype extensions, standalone stream functions, SSE structures, termination handling, OpenAI streaming examples, and React UI update snippets.
+ - `references/api.md`: prototype extensions, standalone converters, SSE field parsing rules, termination, result extractors and decorator usage. Load it for exact signatures or custom extractors.
## Related Skills
- - $fetcher-openai-client: Use for higher-level OpenAI chat client setup.
- - $fetcher-decorator-service: Use when the streaming endpoint is declared with decorators.
- - $fetcher-react-hooks: Use when streaming data drives React state.
+ - $fetcher-openai-client: typed OpenAI chat completions with `[DONE]` handled.
+ - $fetcher-decorator-service: streaming endpoints declared with decorators.
+ - $fetcher-integration: the Fetcher that produces the `Response`.