--- name: openproxy-chat description: Send chat completions through OpenProxy's OpenAI-compatible API — POST /v1/chat/completions, the `provider/model` addressing convention, combos as bare model names, `stream: true` SSE framing, tools / tool_calls passthrough, and reading combo fallback out of a response. Use whenever the user asks to call a model through the proxy, pick a model from GET /v1/models, wire an agent loop to the router, or debug a chat call that 404s with `No active credentials for provider: …`, 400s with `Unknown combo model`, or returns an unexpected `model` field. --- # openproxy-chat — chat completions through `/v1/chat/completions` OpenProxy exposes an OpenAI-compatible chat surface on `127.0.0.1:4623`. One endpoint — `POST /v1/chat/completions` — fronts 40+ providers, with model-level account fallback, combos, and request/response format translation underneath. This skill is about getting a chat call right: addressing a model, streaming it, using tools, and reading back what the router actually did. ## 0 · Preconditions The server must already be running. Check before anything else: ```bash curl -sS http://127.0.0.1:4623/health ``` `/health` needs no auth and returns `{"status":"ok", …}`. If it is not `ok`, start the server first — see the `openproxy` skill (`.agents/skills/openproxy/SKILL.md`) for install and start. Two different credentials, and this trips people up: | Surface | Credential | Notes | |---|---|---| | `/v1/*` (`/v1/chat/completions`, `/v1/models`) | Proxy API key | `Authorization: Bearer $OPENPROXY_API_KEY`. Mint with `openproxy key add --auto`. | | `/api/*` (dashboard, settings, combos, usage, logs) | Admin / management key | The key printed once by `openproxy server init`. A proxy key gets `401 invalid_api_key` here. | For the `curl` examples in this skill you need the **proxy** key: ```bash export BASE=http://127.0.0.1:4623 export OPENPROXY_API_KEY="$(openproxy key list --robot | jq -r '.data.keys[0].key')" ``` `openproxy key list` masks secrets in human output; `--robot` emits the full `key` value (handle it as a secret — do not paste it into a commit, a log, or a shared transcript). To mint a fresh one instead: `openproxy key add my-agent --auto`. The admin key from `server init` also authenticates `/v1/*`, so one credential can cover both surfaces. Sections that touch `/api/*` — the `openproxy chat send|stream`, `openproxy usage …`, and `openproxy logs …` commands — need the **admin** key specifically. If `/v1/*` answers an unauthenticated request with `404` instead of `401`, the install has `requireApiKey` off — `/v1` is open on loopback. That is a settings choice, not a bug; see "Auth" in §8. ## 1 · Pick a model from `GET /v1/models` Never guess a model id. Read the list the router is actually advertising: ```bash curl -sS "$BASE/v1/models" -H "Authorization: Bearer $OPENPROXY_API_KEY" ``` ```json { "object": "list", "data": [ {"id": "ollama/glm-4.7-flash", "object": "model", "owned_by": "ollama", "root": "glm-4.7-flash"}, {"id": "dbg-combo", "object": "model", "owned_by": "combo", "root": "dbg-combo"} ] } ``` `owned_by` tells you which of the two addressing forms applies: - `owned_by: "combo"` → a **combo**. Use the id **bare**: `"model": "dbg-combo"`. - anything else → a **provider model**. Use it with its prefix: `"model": "ollama/glm-4.7-flash"`. Only models that are *active* and *not disabled* on the Providers page appear here, so a model you know exists but do not see is disabled — enable it in the dashboard under `http://127.0.0.1:4623/dashboard/providers`. Resolve any single id to see exactly what it routes to: ```bash curl -sS "$BASE/v1/models/info?id=ollama/glm-4.7-flash" -H "Authorization: Bearer $OPENPROXY_API_KEY" # {"id":"ollama/glm-4.7-flash","provider":"ollama","model":"glm-4.7-flash", # "routeKind":"Direct","owned_by":"ollama","kind":"llm","endpoint":"/v1/chat/completions", …} curl -sS "$BASE/v1/models/info?id=dbg-combo" -H "Authorization: Bearer $OPENPROXY_API_KEY" # {"id":"dbg-combo","provider":null,"model":"dbg-combo","routeKind":"Combo","owned_by":""} ``` `routeKind` is the ground truth for §3. `endpoint` is the kind→path table: an `llm` or `imageToText` model posts to `/v1/chat/completions`; an `embedding` model posts to `/v1/embeddings`; `tts` → `/v1/audio/speech`; `image` → `/v1/images/generations`. Filter by kind with `GET /v1/models/` — `image`, `tts`, `stt`, `embedding`, `image-to-text`, `web`. There is deliberately **no `llm` arm** in `list_models_by_kind` (`src/server/api/v1_models.rs`), so `GET /v1/models/llm` 404s: an unrecognised kind is resolved as a model *id* lookup. Use `GET /v1/models` (no kind), which returns `llm` only. ## 2 · Model addressing — the prefix convention The `model` string is resolved by `get_model_info` (`src/core/model/mod.rs`). The rules, in order: 1. **`combo:`** → always routed as a combo, even if `` is also a provider model. This is the unambiguous form; use it whenever a name is ambiguous. 2. **`/`** → routed directly. The prefix is the provider id (e.g. `ollama`, `anthropic`, `openai`). For a custom provider node the prefix is the node's **prefix**, and credentials are keyed by the node's **UUID** — see `get_model_info` and the note in the `openproxy` skill §4. 3. **bare name** → resolved as, in order: - a **combo** with that exact name, or - a **model alias** from settings, or - a built-in alias (`grok-build`), or - a last-resort provider guess from the family prefix: `claude-*` → `anthropic`, `gemini-*` → `gemini`, `gpt-*` / `o1` / `o3` / `o4` → `openai`, `deepseek-*` → `openrouter`, everything else → `openai`. Step 3's fallback is why an unqualified `gpt-4o-mini` reaches `openai` even when you only have credentials for something else. **Always pass the `provider/model` form** unless you mean a combo. The model string also strips a few annotations before resolution: Claude Code's `[1m]` context marker, and a `(level)` / `-level` thinking suffix. The real 1M capability travels in the `anthropic-beta` header, which is forwarded untouched. ## 3 · The request Minimum body: ```bash curl -sS "$BASE/v1/chat/completions" \ -H "Authorization: Bearer $OPENPROXY_API_KEY" \ -H 'content-type: application/json' \ -d '{ "model": "ollama/glm-4.7-flash", "messages": [{"role": "user", "content": "Reply with exactly: pong"}] }' ``` ```json { "id": "chatcmpl-…", "object": "chat.completion", "model": "glm-4.7-flash", "choices": [ {"index": 0, "message": {"role": "assistant", "content": "pong"}, "finish_reason": "stop"} ], "usage": {"prompt_tokens": 12, "completion_tokens": 2, "total_tokens": 14} } ``` A missing or empty `model` is a `400 {"error":{"code":"bad_request","message":"Missing model"}}`. The same handler is also mounted at `/chat/completions` and `/v1/v1/chat/completions`, so a client whose base URL is already the bare host (or already ends in `/v1`) still lands on one code path. `/v1/chat/completions` is the canonical path — use it. ### The same call from the CLI `openproxy chat` drives the running server. It requires a live server — there is no offline mode. ```bash openproxy chat models # GET /v1/models, first 20 ids + count openproxy chat send --model ollama/glm-4.7-flash --prompt "ping" echo "ping" | openproxy chat send --model my-combo --prompt - openproxy chat send --model my-combo --prompt "hi" --system "be terse" openproxy chat stream --model my-combo --prompt "hi" # streams until Ctrl+C ``` `--prompt` defaults to `-`, so piping on stdin is the default path. Add `--robot` to any of these for a stable `openproxy.v1.*` envelope instead of human output — `chat stream --robot` emits one NDJSON line per chunk with schema `openproxy.v1.chat.event`, and `chat send --robot` emits `openproxy.v1.chat.response`. **Auth differs by subcommand, which is a real trap.** `chat models` hits `/v1/models` and works with a proxy API key. `chat send` and `chat stream` post to `/api/dashboard/chat/completions`, which is guarded by the **admin/management key** — with only a proxy key they fail `401 invalid_api_key`. That dashboard surface also normalizes a bare model name by prefixing it when exactly one active connection advertises that model. For scripted work against a different host, or to control the exact bytes on the wire, use `curl` against `/v1/chat/completions` with a proxy key. ## 4 · Streaming Set `"stream": true`. The response comes back as `text/event-stream` with OpenAI SSE framing: ``` data: {"choices":[{"delta":{"content":"hel"},"index":0}]} data: {"choices":[{"delta":{"content":"lo"},"index":0}]} data: [DONE] ``` ```bash curl -N -sS "$BASE/v1/chat/completions" \ -H "Authorization: Bearer $OPENPROXY_API_KEY" \ -H 'content-type: application/json' \ -d '{"model":"ollama/glm-4.7-flash","messages":[{"role":"user","content":"count to 3"}],"stream":true}' ``` Things that bite: - **Frames are `data: ` prefixed and blank-line separated; the stream ends with a literal `data: [DONE]`.** A client that splits on newlines instead of blank lines will break. - **Streaming is the default when `stream` is absent.** `resolve_stream_flags` sets `stream = (body.stream != false)`. Send `"stream": false` explicitly for a buffered JSON reply. - **Some providers force streaming upstream even when you did not ask.** When that happens the router aggregates the SSE and returns a single JSON body (`sse_to_json`), so the shape you get back still matches what you requested. - **`Accept: application/json` flips a non-`stream:true` request to non-streaming** — but an explicit `"stream": true` always wins. - **Gemini / Antigravity / GeminiCli sources always stream**, whatever `stream` says. - Image-generation models on `antigravity` / `gemini-cli` are forced non-streaming. ## 5 · Tools and `tool_calls` `tools` and `tool_calls` pass through the OpenAI dialect unchanged, and are **translated** when the target provider speaks a different dialect (Claude `tools` / `tool_use`, Gemini `functionDeclarations`, Kiro, Cursor, Codex, Ollama, CommandCode). You send the OpenAI shape; the router handles the rest. Declare tools: ```bash curl -sS "$BASE/v1/chat/completions" \ -H "Authorization: Bearer $OPENPROXY_API_KEY" \ -H 'content-type: application/json' \ -d '{ "model": "ollama/glm-4.7-flash", "messages": [{"role": "user", "content": "What is the weather in Osaka?"}], "tools": [{ "type": "function", "function": { "name": "get_weather", "description": "Get current weather for a city", "parameters": { "type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"] } } }], "tool_choice": "auto" }' ``` The assistant replies with `choices[0].message.tool_calls` — an array of `{id, type: "function", function: {name, arguments}}` where `arguments` is a **JSON string**, not a parsed object. Parse it before using it. Feed the result back as a `role: "tool"` message carrying the matching `tool_call_id`: ```json { "model": "ollama/glm-4.7-flash", "messages": [ {"role": "user", "content": "What is the weather in Osaka?"}, {"role": "assistant", "tool_calls": [ {"id": "call_abc", "type": "function", "function": {"name": "get_weather", "arguments": "{\"city\":\"Osaka\"}"}} ]}, {"role": "tool", "tool_call_id": "call_abc", "content": "18C, light rain"} ], "tools": [ /* re-send the same tools array */ ] } ``` Re-sending `tools` on the follow-up turn is required — the router has no memory of the previous request. In a stream, tool arguments arrive fragmented across `delta.tool_calls[].function.arguments`; the `index` field pairs fragments across chunks. Concatenate before parsing. Empty `delta.tool_calls: []` arrays — which some providers emit in every chunk — are stripped before reaching you, so a `!= null` check on `delta.tool_calls` behaves as OpenAI SDKs expect. Two provider-specific behaviors, both intentional: - **TTS models drop `tools` and every `role: "tool"` message** before dispatch. - **Claude-dialect clients get their tools de-duplicated** before dispatch. ## 6 · Combos and fallback A **combo** is a named list of `provider/model` entries dispatched as one. It is addressed as a bare model name, and is what makes a chat call survive a provider going down mid-session. ```bash openproxy combo list # existing combos openproxy combo get my-combo # membership openproxy combo test my-combo # dry-run the expansion, verify reachability openproxy combo create --name my-stack \ --models "openai/gpt-4o,anthropic/claude-3-5-sonnet" \ --strategy fallback # fallback (default) | round-robin | sticky-round-robin ``` Call it with the combo name, bare or explicit: ```bash -d '{"model":"my-stack","messages":[{"role":"user","content":"ping"}]}' -d '{"model":"combo:my-stack","messages":[{"role":"user","content":"ping"}]}' ``` A name that is neither a combo, an alias, nor a prefixed provider model does **not** produce "Unknown combo model" by accident — that 400 only comes from the `combo:` form naming a combo that does not exist or is disabled: ```bash # 400 {"error":{"code":"bad_request","message":"Unknown combo model"}} -d '{"model":"combo:no-such-combo", …}' ``` ### How fallback shows up in the response When a combo member fails, the router tries the next member. **The response body's `model` field names the member that actually answered**, not the combo you asked for. That is the primary signal: ``` request "model": "my-stack" → response "model": "gpt-4o" (second member served it) ``` There is no separate "which member served" field or header — read `model`. To see the full picture per request, check usage. The history rows carry `provider` and `model` per request, so a combo's members appear as separate rows: ```bash openproxy usage history --limit 20 # model / provider / cost per row openproxy usage history --limit 20 --robot # same, as openproxy.v1.usage.history openproxy usage logs # paginated combined activity log openproxy logs tail --follow # live NDJSON log stream ``` These talk to `/api/usage/*` and `/api/observability/*`, so they need the **admin/management key**, not a proxy key — with only a proxy key they return `401 invalid_api_key`. Export the right one first: `export OPENPROXY_API_KEY="$(openproxy --robot server init | jq -r '.data.admin_key.key')"`. (No code writes `~/.openproxy/admin.key`; only the bootstrap skill tells you to create it by hand, so reading it silently yields an empty key and a cascade of 401s.) `GET /v1/usage` and `GET /api/usage/history` serve the same data over HTTP. A response that carries no token usage at all (most SSE streams) records **no** usage row — absence of a row is not proof the call did not happen. When every member of a combo fails, the members that errored are auto-quarantined with a cooldown drawn from the failure class, so the *next* request skips them rather than re-attempting a known-broken member. That is why a failing combo gets *faster* to error, not slower, on repeat calls. ## 7 · Error reference Every error is the OpenAI envelope: `{"error": {"message", "type", "code"}}`. Chat-specific arms: | Status | `error.code` | `error.message` | Meaning | |---|---|---|---| | 400 | `bad_request` | `Missing model` | `model` absent or empty. | | 400 | `bad_request` | `Unknown combo model` | `combo:` names a combo that does not exist or is disabled. | | 401 | `invalid_api_key` | — | Bad/expired bearer on a `/api/*` surface. | | 403 | _(upstream's)_ | provider's own wording | Passed through from the provider — quota, entitlement, or account ban. | | 404 | `model_not_found` | `No active credentials for provider: ` | See below. | | 429 | _(upstream's)_ | provider's own wording | Passed through from the provider. | | 502 | _(upstream's)_ | provider's own wording | The provider **has** a credential and failed while serving. | | 503 | `service_unavailable` | `[/] (reset after 4m 12s)` | Every account for that provider is rate-limited / cooling down. `Retry-After` header is set. | | 503 | `service_unavailable` | `All accounts for / are unavailable` | Accounts were tried and all exhausted. | Note the split: the `400` / `404` / `503` rows are **built by OpenProxy**, so their `code` and `message` are stable. The `403` / `429` / `502` rows carry the **upstream provider's** body verbatim (served back as JSON), so their `code` and `message` are whatever that provider emits and will not match this table. ### `404 No active credentials for provider: X` — what it actually means Verified verbatim against a running server: ```bash $ curl -sS -w '\nHTTP %{http_code}\n' "$BASE/v1/chat/completions" \ -H "Authorization: Bearer $OPENPROXY_API_KEY" -H 'content-type: application/json' \ -d '{"model":"totally-unknown-provider/gpt-4o-mini","messages":[{"role":"user","content":"hi"}]}' {"error":{"code":"model_not_found","message":"No active credentials for provider: totally-unknown-provider","type":"invalid_request_error"}} HTTP 404 ``` Despite the 404 and the `model_not_found` code, **this is not a routing or URL problem** — the model string resolved fine. The precise meaning is: *for that provider, no connection was ever attempted.* The router picks the status from what the credential lookup reported, not from the model name: - **nothing excluded** (no account was ever tried) → `404 No active credentials for provider: X`. There is no usable credential on file for `X` at all. - **every account rate-limited / cooling down** → `503` + `Retry-After`, message `[/] (reset after 4m 12s)`. - **accounts were tried and all failed** → `503 All accounts for / are unavailable`. So when you see the 404, check the *provider*, not the model: 1. Is the prefix spelled right? A typo'd prefix is a provider that does not exist — same 404. Use `GET /v1/models` and copy an id verbatim. 2. Does that provider have a connection? `openproxy provider list`, or `GET /api/providers` with the **admin** key. 3. Is the connection active and does it hold a credential? A connection with `api_key: null` — an OAuth provider whose token has not been refreshed, or an API-key provider never filled in — is exactly this case. The request never reaches the upstream; the provider receives zero traffic. 4. Was the model disabled on the Providers page? That removes it from `/v1/models`, but a directly addressed `provider/model` still resolves — so a disabled model gives a *provider* error, not a 404 about the model. Note the asymmetry: **the same provider with a credential that then fails gives a `502`, not this 404.** If you are getting 404, adding credentials is the fix; debugging the upstream key is not. ## 8 · Auth `/v1/chat/completions` requires a valid, **active** proxy API key only when the `requireApiKey` setting is on — which is the default on a fresh install, and which falls back to `requireLogin` on installs predating the field. When it is off, `/v1` accepts unauthenticated loopback requests (and returns `404` for an unconfigured provider rather than `401`, since there is no auth check to fail). ```bash openproxy key list # existing keys (secrets masked) openproxy key add my-agent --auto # mint a fresh op-… secret, prints it once openproxy key disable my-agent # revoke without deleting ``` Auth is read from `Authorization: Bearer `, or `x-api-key: `, or `x-goog-api-key: ` (Gemini-style clients). `Authorization: Bearer …` is the one to use. Forwarded headers (`X-Forwarded-For`, `X-Real-IP`, `X-Forwarded-Proto`, …) are **stripped** unless `TRUST_PROXY=true`, because on a default single-process deploy they are spoofable and are used for rate limiting and IP logging. ## 9 · Verifications Run these in order. Each one isolates a different layer. ```bash # 1. Server is up (no auth) curl -sS "$BASE/health" | head -c 80 # 2. Endpoint advertises itself (no auth) curl -sS "$BASE/v1" # → {"version":"v1","endpoints":["/v1/chat/completions", …]} # 3. Your bearer works on /v1 curl -sS -o /dev/null -w '%{http_code}\n' \ "$BASE/v1/models" -H "Authorization: Bearer $OPENPROXY_API_KEY" # 4. The model you picked is real, and you know what it resolves to curl -sS "$BASE/v1/models/info?id=my-model-or-combo" \ -H "Authorization: Bearer $OPENPROXY_API_KEY" # 5. Non-streaming round trip curl -sS "$BASE/v1/chat/completions" \ -H "Authorization: Bearer $OPENPROXY_API_KEY" -H 'content-type: application/json' \ -d '{"model":"my-model","messages":[{"role":"user","content":"ping"}],"stream":false}' # 6. Streaming round trip — must end with data: [DONE] curl -N -sS "$BASE/v1/chat/completions" \ -H "Authorization: Bearer $OPENPROXY_API_KEY" -H 'content-type: application/json' \ -d '{"model":"my-model","messages":[{"role":"user","content":"ping"}],"stream":true}' # 7. The request was actually recorded openproxy usage history --limit 5 ``` If step 5 returns a body but no `choices`, the failure is upstream — read `error.message`, it carries the provider's own wording with the router's prefixes (`Error from provider:`, `Upstream error:`) already stripped. If step 5 returns `404 No active credentials for provider: X`, go to §7. To exercise the whole path **without a live provider**, the `x-openproxy-sim: mock` request header short-circuits dispatch and renders a canned response in your request's own dialect. Any `x-openproxy-sim-*` header bypasses the response cache, so a simulated reply can never be served to a real request: ```bash curl -sS "$BASE/v1/chat/completions" \ -H "Authorization: Bearer $OPENPROXY_API_KEY" -H 'content-type: application/json' \ -H 'x-openproxy-sim: mock' \ -d '{"model":"ollama/glm-4.7-flash","messages":[{"role":"user","content":"hi"}]}' ``` Note the mock still resolves credentials and the model, so it will not mask a `404` or a `400`. ## 10 · Debugging an unexpected `model` field The `model` you sent and the `model` you got back can differ for four distinct reasons. Check them in this order: 1. **You sent a bare name and it resolved by inference.** `gpt-4o-mini` → `openai`; `claude-*` → `anthropic`. Send `provider/model` explicitly to remove the guesswork. 2. **You sent a combo.** The response names the member that served. This is correct behavior — see §6. 3. **The catalog remaps the model upstream.** A model with an `upstreamModelId` in the provider catalog is dispatched under that id, and the provider echoes it. `GET /v1/models/info?id=…` returns `upstreamModelId` when one is set. 4. **The provider echoed a different name than the one you sent** — common with OpenRouter-style gateways that report a canonical or dated model id. Nothing to fix. ## When _not_ to use this skill - **Wiring a CLI tool** (Claude Code, Codex, Cursor, Cline) to the proxy — that is base-URL + bearer config, not chat semantics. See `.agents/skills/openproxy-cli-tools/SKILL.md`. - **Combo construction, membership, strategies, round-robin vs fallback** — this skill covers calling a combo, not designing one. See `.agents/skills/openproxy-combos/SKILL.md`, and use `openproxy combo test ` to verify an expansion end to end. - **Non-chat media** — images, TTS, STT, embeddings, web search, web fetch each have their own endpoint and skill under `.agents/skills/`. - **Installing or starting the server** — see `.agents/skills/openproxy/SKILL.md`. - **Anthropic-native (`/v1/messages`) or Responses-API (`/v1/responses`) clients** — those are separate handlers; the OpenAI dialect documented here is translated to whatever the target provider needs, so an OpenAI-shaped body is the portable choice. ## See also - `.agents/skills/openproxy/SKILL.md` — install, start, auth bootstrap - `.agents/skills/openproxy-combos/SKILL.md` — building and editing combos - `docs/parity-9router.md` — intentional divergences from 9router, and the pipeline order - `GET $BASE/v1` — the live endpoint list, on the running server - `openproxy chat --help` / `openproxy chat --help`