llm-layer · diff
git:20260319.243527c to git:20260319.c97e02c
5 added, 2 removed. Audit A to A.
What this update changes in the checks
No check changed: every fixed check gives the same answer before and after this update.
A tripped check is a label, not a verdict: read the matched text and the lines below. This week's updates that newly trip a check.
---
description: LLM provider layer — provider resolution, streaming, JSON parsing for caliber
globs: src/llm/**/*.ts,src/ai/**/*.ts
alwaysApply: false
---
- All LLM calls go through `src/llm/index.ts` — never import @anthropic-ai/sdk, openai, or @anthropic-ai/vertex-sdk directly
- `llmCall()` returns plain text; `llmJsonCall<T>()` returns parsed+typed JSON
- Both auto-retry on transient errors via `TRANSIENT_ERRORS` array with exponential backoff (`1000 * Math.pow(2, attempt - 1)`)
- Provider resolution: `ANTHROPIC_API_KEY` → `VERTEX_PROJECT_ID` → `OPENAI_API_KEY` → `CALIBER_USE_CURSOR_SEAT` → `CALIBER_USE_CLAUDE_CLI` → config file
- - Each provider implements `LLMProvider` interface from `src/llm/types.ts`: `call()` + `stream()` + optional `listModels()`
+ - Each provider implements `LLMProvider` interface from `src/llm/types.ts`: `call(options: LLMCallOptions): Promise<string>` + `stream(options, callbacks)`
- For streaming: `getProvider()` then `provider.stream(options, { onText, onEnd, onError })` — `onEnd` receives optional `meta?: { stopReason?, usage? }`
- JSON parsing: `extractJson()` for bracket-balancing, `parseJsonResponse()` for full pipeline — never raw `JSON.parse()` on LLM output
- Fast model: `getFastModel()` from `src/llm/config.ts` — spread `...(fastModel ? { model: fastModel } : {})` into call options
- - Use `parseSeatBasedError()` from `src/llm/seat-based-errors.ts` for both Cursor and Claude CLI error parsing
+ - Token usage tracking: call `trackUsage(model, usage)` from `src/llm/usage.ts` after each provider response
+ - Use `parseSeatBasedError(stderr, exitCode)` from `src/llm/seat-based-errors.ts` for Cursor and Claude CLI error detection
+ - Model recovery: `isModelNotAvailableError()` + `handleModelNotAvailable()` from `src/llm/model-recovery.ts`
- Seat-based providers (`cursor`, `claude-cli`) need `stdio: ['pipe', 'pipe', 'pipe']` in spawn — never `inherit` for stderr
- Call `validateModel()` before long/streaming work in `init`, `regenerate`, `learn`, `refresh`
+ - Cursor `agent --print --output-format stream-json`: events are `assistant` deltas with `timestamp_ms`, then `result` — skip duplicates by checking `timestamp_ms` presence