llm-layer · git:20260319.c97e02c · 2026-03-19 · sha256 9c462dee7a336aef
llm-layer git:20260319.c97e02cA
Immutable. This exact content never changes and is served at /api/v1/blob/9c462dee7a336aef.
---
description: LLM provider layer — provider resolution, streaming, JSON parsing for caliber
globs: src/llm/**/*.ts,src/ai/**/*.ts
alwaysApply: false
---
- All LLM calls go through `src/llm/index.ts` — never import @anthropic-ai/sdk, openai, or @anthropic-ai/vertex-sdk directly
- `llmCall()` returns plain text; `llmJsonCall<T>()` returns parsed+typed JSON
- Both auto-retry on transient errors via `TRANSIENT_ERRORS` array with exponential backoff (`1000 * Math.pow(2, attempt - 1)`)
- Provider resolution: `ANTHROPIC_API_KEY` → `VERTEX_PROJECT_ID` → `OPENAI_API_KEY` → `CALIBER_USE_CURSOR_SEAT` → `CALIBER_USE_CLAUDE_CLI` → config file
- Each provider implements `LLMProvider` interface from `src/llm/types.ts`: `call(options: LLMCallOptions): Promise<string>` + `stream(options, callbacks)`
- For streaming: `getProvider()` then `provider.stream(options, { onText, onEnd, onError })` — `onEnd` receives optional `meta?: { stopReason?, usage? }`
- JSON parsing: `extractJson()` for bracket-balancing, `parseJsonResponse()` for full pipeline — never raw `JSON.parse()` on LLM output
- Fast model: `getFastModel()` from `src/llm/config.ts` — spread `...(fastModel ? { model: fastModel } : {})` into call options
- Token usage tracking: call `trackUsage(model, usage)` from `src/llm/usage.ts` after each provider response
- Use `parseSeatBasedError(stderr, exitCode)` from `src/llm/seat-based-errors.ts` for Cursor and Claude CLI error detection
- Model recovery: `isModelNotAvailableError()` + `handleModelNotAvailable()` from `src/llm/model-recovery.ts`
- Seat-based providers (`cursor`, `claude-cli`) need `stdio: ['pipe', 'pipe', 'pipe']` in spawn — never `inherit` for stderr
- Call `validateModel()` before long/streaming work in `init`, `regenerate`, `learn`, `refresh`
- Cursor `agent --print --output-format stream-json`: events are `assistant` deltas with `timestamp_ms`, then `result` — skip duplicates by checking `timestamp_ms` presence