adding-models · diff
git:20251222.899966f to git:20260822.1f4d3e3
17 added, 32 removed. Audit A to A.
---
name: adding-models
- description: Guide for adding new LLM models to Letta Code. Use when the user wants to add support for a new model, needs to know valid model handles, or wants to update the model configuration. Covers models.json configuration, CI test matrix, and handle validation.
+ description: Guide for adding new LLM models to Letta Code. Use when the user wants to add support for a new model, needs to know valid model handles, or wants to update model-specific compatibility behavior. Covers runtime catalog sources, CI test matrices, and handle validation.
---
# Adding Models
This skill guides you through adding a new LLM model to Letta Code.
## Quick Reference
**Key files**:
- - `src/models.json` - Model definitions (required)
+ - `src/agent/remote-model-catalog.ts` - Runtime catalog loading and projection
+ - `src/agent/model-catalog.ts` - Model lookup and compatibility aliases
- `.github/workflows/ci.yml` - CI test matrix (optional)
- `src/tools/manager.ts` - Toolset detection logic (rarely needed)
## Workflow
### Step 1: Find Valid Model Handles
- Query the Letta API to see available models:
+ Query the hosted catalog to see preset IDs and handles:
```bash
+ curl -s https://api.letta.com/v1/models/catalog | jq '.models[] | [.id, .handle]'
+ ```
+
+ To inspect the models currently available from an API backend, query its model inventory:
+
+ ```bash
curl -s https://api.letta.com/v1/models/ | jq '.[] | .handle'
```
- Or filter by provider:
+ Or filter the inventory by provider:
```bash
curl -s https://api.letta.com/v1/models/ | jq '.[] | select(.handle | startswith("google_ai/")) | .handle'
```
Common provider prefixes:
- `anthropic/` - Claude models
- `openai/` - GPT models
- `google_ai/` - Gemini models
- `google_vertex/` - Vertex AI
- `openrouter/` - Various providers
- ### Step 2: Add to models.json
+ ### Step 2: Update the Owning Catalog
- Add an entry to `src/models.json`:
+ Letta Code does not bundle a model catalog:
- ```json
- {
- "id": "model-shortname",
- "handle": "provider/model-name",
- "label": "Human Readable Name",
- "description": "Brief description of the model",
- "isFeatured": true, // Optional: shows in featured list
- "updateArgs": {
- "context_window": 180000,
- "temperature": 1.0 // Optional: provider-specific settings
- }
- }
- ```
+ - API and hosted presets come from the server's `GET /v1/models/catalog` response.
+ - Local model inventory comes from pi-ai and the active provider runtimes.
- **Field reference**:
- - `id`: Short identifier used with `--model` flag (e.g., `gemini-3-flash`)
- - `handle`: Full provider/model path from the API (e.g., `google_ai/gemini-3-flash-preview`)
- - `label`: Display name in model selector
- - `description`: Brief description shown in selector
- - `isFeatured`: If true, appears in featured models section
- - `updateArgs`: Model-specific configuration (context window, temperature, reasoning settings, etc.)
+ Add the model at the source that owns it. A hosted preset belongs in the server catalog. A local provider model belongs in pi-ai or that provider's discovery runtime.
- **Provider prefixes**:
- - `anthropic/` - Anthropic (Claude models)
- - `openai/` - OpenAI (GPT models)
- - `google_ai/` - Google AI (Gemini models)
- - `google_vertex/` - Google Vertex AI
- - `openrouter/` - OpenRouter (various providers)
+ Only change this repository when the model needs Letta Code-specific compatibility behavior, such as preserving an established CLI alias or recognizing a new provider for toolset selection. Keep that logic narrow and derive the handle and metadata from the runtime catalog rather than copying model definitions here.
### Step 3: Test the Model
Test with headless mode:
```bash
bun run src/index.ts --new --model <model-id> -p "hi, what model are you?"
```
Example:
```bash
bun run src/index.ts --new --model gemini-3-flash -p "hi, what model are you?"
```
### Step 4: Add to CI Test Matrix (Optional)
To include the model in automated testing, add it to `.github/workflows/ci.yml`:
```yaml
# Find the headless job matrix around line 122
model: [gpt-5-minimal, gpt-4.1, sonnet-4.5, gemini-pro, your-new-model, glm-4.6, haiku]
```
## Toolset Detection
Models are automatically assigned toolsets based on provider:
- `openai/*` → `codex` toolset
- `google_ai/*` or `google_vertex/*` → `gemini` toolset
- Others → `default` toolset
This is handled by `isGeminiModel()` and `isOpenAIModel()` in `src/tools/manager.ts`. You typically don't need to modify this unless adding a new provider.
## Common Issues
**"Handle not found" error**: The model handle is incorrect. Run the validation script to see valid handles.
**Model works but wrong toolset**: Check `src/tools/manager.ts` to ensure the provider prefix is recognized.