amrouter · diff

git:20260727.056e3f8 to git:20260911.fa4fe6b

1 added, 1 removed. Audit A to A.

---

description: >-

Self-hosted AI gateway with one OpenAI-compatible endpoint for multi-provider LLM, embedding,

image, and audio routing, automatic fallback, load balancing, and cost optimization.

---

# AMRouter Skill for AI Agents

**Self-hosted AI gateway: one endpoint, many providers, auto-fallback.**

AMRouter is an OpenAI-compatible API gateway that routes requests across dozens of LLM, embedding, image, and audio providers with automatic load balancing, fallback, and cost optimization—designed for agents to access multi-provider AI infrastructure without provider lock-in.

---

## Quick Links

- **Official Site:** https://github.com/ahwanulm/AMRouter

- **Documentation:** https://github.com/ahwanulm/AMRouter/tree/main/docs

- **Features:** Multi-provider routing, auto-fallback, Cloudflare Workers AI, OpenAI-compatible REST API

---

## What is AMRouter?

AMRouter v2 is a rewrite of 9Router with clean separation:

- **Express backend** — Routes requests to 50+ LLM, embedding, image, audio, and utility providers

- **Vite + React frontend** — Dashboard to manage providers, connections, proxy pools, CLI tools, automation

- **OpenAI-compatible API** — Works with any client that supports `/v1/chat/completions`, `/v1/embeddings`, `/v1/images/generations`, `/v1/audio/speech`, etc.

### Key Features

✅ **Multi-provider routing** — Cloudflare Workers AI, OpenAI, Anthropic Claude, Google Gemini, Groq, Mistral, Cohere, Voyage, GitHub, and 40+ more

✅ **Auto-fallback combos** — Define provider chains (e.g., `vip`, `budget`, `openai-only`); routes fail over automatically

✅ **Cost optimization** — Track quota per provider, use token counters for cost estimation

✅ **Cloudflare Workers AI Automation** — Auto-register accounts, extract keys via Playwright + 2Captcha + temp mail

✅ **Dashboard UI** — Manage providers, connections, automation, view logs and metrics

✅ **Agent Skills** — Pre-built SKILL.md files for Claude, Gemini, and other AI coding agents

✅ **OpenAI SDK compatible** — Works with OpenAI npm client, Python client, or any REST client

---

## When to Use

### ✅ Use AMRouter if you:

- Want to **consolidate multi-provider access** into one API endpoint

- Need **automatic fallback** when a provider fails or runs out of quota

- Want **cost tracking** across providers (quota limits, token counting)

- Operate **multiple LLM models** and need a unified interface

- Are building **agent infrastructure** and want easy provider swapping

- Want **Cloudflare Workers AI** automation (auto-registration, key extraction)

- Need a **self-hosted** solution (not vendor-managed unblockers)

### ❌ Don't use AMRouter if you:

- Only need one provider (just use their SDK directly)

- Can't run a Node.js backend

- Don't need fallback or multi-provider orchestration

- Want to use a managed service like Anthropic's API directly

---

## Installation & Setup

### Step 1: Requirements

- Node.js 20+

- Python 3.10+ (for automation features)

- Chromium (for Playwright automation; optional)

### Step 2: Install

```bash

git clone https://github.com/ahwanulm/AMRouter.git

cd AMRouter

npm install

```

### Step 3: Configure Environment

```bash

cp backend/.env.template backend/.env

```

Key environment variables:

| Variable | Description | Example |

|---|---|---|

| `PORT` | Backend server port | `3001` |

| `REQUIRE_LOGIN` | Enable authentication | `true` |

| `JWT_SECRET` | Secret for JWT signing | (generate a random string) |

| `ADMIN_PASSWORD` | Dashboard admin password | (set securely) |

| `OPENAI_API_KEY` | OpenAI provider key | (optional) |

| `ANTHROPIC_API_KEY` | Anthropic provider key | (optional) |

| `GOOGLE_API_KEY` | Google Gemini key | (optional) |

### Step 4: Development

```bash

npm run dev # Start backend + frontend concurrently

npm run backend # Backend only (port 3001)

npm run frontend # Frontend only (port 5177)

```

### Step 5: Production

```bash

cd frontend && npm run build

cd ..

NODE_ENV=production npm start

```

Server runs on port 3001 (configurable). Frontend is built into `backend/public/`.

---

## Core Concepts

### 1. Providers

A **provider** is an LLM, embedding, image, or audio service (OpenAI, Anthropic, Gemini, etc.).

Each provider has:

- **Key/API credentials** — managed via dashboard

- **Models** — specific model IDs (e.g., `openai/gpt-4o`, `anthropic/claude-3-opus`)

- **Quota** — optional limits on tokens or requests

- **Priority** — used in fallback chains

### 2. Combos (Fallback Chains)

A **combo** is a named chain of providers. When you call `/v1/chat/completions` with combo name `vip`, it tries:

1. Provider A

2. If A fails or quota exceeded → Provider B

3. If B fails → Provider C

4. Etc.

Example combo config:

```json

{

"name": "vip",

"providers": ["openai", "anthropic", "gemini"],

"strategy": "round-robin"

}

```

### 3. OpenAI-Compatible API

AMRouter exposes the same endpoints as OpenAI:

| Endpoint | Models | Example |

|---|---|---|

| `POST /v1/chat/completions` | Chat models | `openai/gpt-4o`, `anthropic/claude-3-opus` |

| `POST /v1/embeddings` | Embedding models | `openai/text-embedding-3-small`, `gemini/text-embedding-004` |

| `POST /v1/images/generations` | Image models | `openai/dall-e-3`, `openai/dall-e-2` |

| `POST /v1/audio/speech` | TTS models | `openai/tts-1`, `elevenlabs/eleven_monolingual_v1` |

| `POST /v1/audio/transcriptions` | STT models | `openai/whisper-1` |

| `GET /v1/models` | List all models | Returns `{data: [...]}` |

---

## Usage Patterns

### Pattern 1: Agent Chat via OpenAI SDK

Use AMRouter as a drop-in OpenAI SDK replacement:

```javascript

import OpenAI from "openai";

const client = new OpenAI({

baseURL: `${process.env.AMROUTER_URL}/v1`,

apiKey: process.env.AMROUTER_KEY || "any-value",

});

const res = await client.chat.completions.create({

model: "openai/gpt-4o", // or combo name like "vip"

messages: [{ role: "user", content: "What is 2+2?" }],

stream: true,

});

for await (const chunk of res) {

process.stdout.write(chunk.choices[0]?.delta?.content || "");

}

```

### Pattern 2: Combo-Based Fallback

Define a fallback combo in dashboard. Agent sends one request:

```bash

curl -X POST ${AMROUTER_URL}/v1/chat/completions \

-H "Authorization: Bearer ${AMROUTER_KEY}" \

-H "Content-Type: application/json" \

-d '{

"model": "budget",

"messages": [{"role": "user", "content": "Summarize this article..."}]

```

AMRouter automatically tries: Groq → Mistral → Gemini (all with cost tracking).

### Pattern 3: Embeddings for RAG

```javascript

const embeddings = await fetch(`${AMROUTER_URL}/v1/embeddings`, {

method: "POST",

headers: {

"Authorization": `Bearer ${AMROUTER_KEY}`,

"Content-Type": "application/json",

body: JSON.stringify({

model: "openai/text-embedding-3-small",

input: ["Document chunk 1", "Document chunk 2"],

}),

});

const { data } = await embeddings.json();

console.log(data[0].embedding); // Vector (1536 dims)

```

### Pattern 4: Image Generation

```bash

curl -X POST ${AMROUTER_URL}/v1/images/generations \

-H "Authorization: Bearer ${AMROUTER_KEY}" \

-H "Content-Type: application/json" \

-d '{

"model": "openai/dall-e-3",

"prompt": "A futuristic city at sunset",

"n": 1,

"size": "1024x1024"

```

### Pattern 5: Cost Tracking & Quota

Query quota endpoint:

```bash

curl ${AMROUTER_URL}/api/quota/providers \

-H "Authorization: Bearer ${AMROUTER_KEY}" | jq '.[]'

```

Output:

```json

[

{

"provider": "openai",

"quotaLimit": 100000,

"quotaUsed": 45230,

"quotaRemaining": 54770,

"costUSD": 12.45

...

]

```

---

## Agent Integration

### With `deep-agents-core`

```javascript

const agentTools = {

amrouter_chat: {

description: "Call any LLM via AMRouter (OpenAI/Anthropic/Gemini/Groq combo fallback)",

input_schema: {

type: "object",

properties: {

model: { type: "string", description: "Provider/combo (openai/gpt-4o, anthropic/claude-3, budget)" },

messages: { type: "array", description: "Chat messages" },

temperature: { type: "number", description: "0–2 (default 1)" },

required: ["model", "messages"],

amrouter_embeddings: {

description: "Generate embeddings via OpenAI/Gemini/Voyage models",

input_schema: {

type: "object",

properties: {

model: { type: "string", description: "openai/text-embedding-3-small, gemini/text-embedding-004, voyage/voyage-large-2" },

input: { type: "string", description: "Text to embed" },

required: ["model", "input"],

};

```

### With `research` Skill

Parallelize multi-model queries:

```bash

# Search across multiple LLM models simultaneously

amrouter-query "Explain quantum computing" \

--models openai/gpt-4o,anthropic/claude-3-opus,gemini/gemini-2-flash \

--parallel \

--compare

```

### With Environment Variables

Set in `.env`:

```bash

AMROUTER_URL=http://localhost:3001

AMROUTER_KEY=sk-amrouter-xxx

```

Then in agent code:

```javascript

const AMROUTER_URL = process.env.AMROUTER_URL;

const AMROUTER_KEY = process.env.AMROUTER_KEY;

// Use in OpenAI SDK as shown above

```

---

## Common Tasks

### Task 1: List Available Providers

```bash

curl ${AMROUTER_URL}/v1/models \

-H "Authorization: Bearer ${AMROUTER_KEY}" | jq '.data[].id'

# Output:

# openai/gpt-4o

# openai/gpt-4-turbo

# anthropic/claude-3-opus-20240229

# google/gemini-2-flash

# groq/mixtral-8x7b-32768

# ...

```

### Task 2: Check Quota for All Providers

```bash

curl ${AMROUTER_URL}/api/quota/providers \

-H "Authorization: Bearer ${AMROUTER_KEY}" | jq '.'

```

### Task 3: Create a Cost-Optimized Combo

Via dashboard:

1. Navigate to **Providers** tab

2. Click **New Combo**

3. Name: `budget`

4. Add providers: Groq, Mistral, Llama 2 (in fallback order)

5. Save

Then agents can use:

```bash

curl -X POST ${AMROUTER_URL}/v1/chat/completions \

-H "Authorization: Bearer ${AMROUTER_KEY}" \

-d '{"model":"budget","messages":[...]}'

```

### Task 4: Stream Responses

```javascript

const res = await fetch(`${AMROUTER_URL}/v1/chat/completions`, {

method: "POST",

headers: {

"Authorization": `Bearer ${AMROUTER_KEY}`,

"Content-Type": "application/json",

body: JSON.stringify({

model: "openai/gpt-4o",

messages: [{ role: "user", content: "Write a poem" }],

stream: true,

}),

});

const reader = res.body.getReader();

while (true) {

const { done, value } = await reader.read();

if (done) break;

const text = new TextDecoder().decode(value);

const lines = text.split("\n");

for (const line of lines) {

if (line.startsWith("data: ")) {

const data = JSON.parse(line.slice(6));

process.stdout.write(data.choices[0]?.delta?.content || "");

}

```

---

- ## Best Practices

+ ## Best practices

### 1. Use Combos for Resilience

Instead of hardcoding a provider:

```javascript

// ❌ Avoid: single provider, no fallback

model: "openai/gpt-4o"

// ✅ Prefer: fallback combo

model: "vip" // Falls back through multiple providers automatically

```

### 2. Respect Quota Limits

Check remaining quota before large requests:

```javascript

const quota = await fetch(`${AMROUTER_URL}/api/quota/providers`).then(r => r.json());

const gpt4oQuota = quota.find(q => q.provider === "openai");

if (gpt4oQuota.quotaRemaining < 10000) {

console.log("Switching to budget model");

model = "budget"; // Fallback to cost-optimized combo

}

```

### 3. Cache Embeddings

Reuse embedding results:

```javascript

const embeddingCache = new Map();

async function getEmbedding(text) {

if (embeddingCache.has(text)) return embeddingCache.get(text);

const res = await fetch(`${AMROUTER_URL}/v1/embeddings`, {

method: "POST",

headers: { "Authorization": `Bearer ${AMROUTER_KEY}`, "Content-Type": "application/json" },

body: JSON.stringify({ model: "openai/text-embedding-3-small", input: text }),

});

const { data } = await res.json();

const embedding = data[0].embedding;

embeddingCache.set(text, embedding);

return embedding;

}

```

### 4. Stream Long Responses

For agents generating long content, use streaming:

```javascript

// Stream large code generation

const res = await client.chat.completions.create({

model: "openai/gpt-4o",

messages: [...],

stream: true, // ✅ Use streaming

});

```

### 5. Log Provider & Cost

Track which provider handled each request:

```javascript

// Extend logging with provider info

const res = await fetch(`${AMROUTER_URL}/v1/chat/completions`, {

method: "POST",

headers: { "Authorization": `Bearer ${AMROUTER_KEY}`, "Content-Type": "application/json" },

body: JSON.stringify({ model: "vip", messages: [...] }),

});

const data = await res.json();

console.log(`Provider: ${data.provider_used || 'unknown'}`); // If AMRouter returns it

console.log(`Cost: $${data.estimated_cost || 'N/A'}`);

```

---

## Performance & Cost

Real benchmarks (vs single-provider setup):

|---|---|---|---|

---

## Troubleshooting

| Issue | Fix |

|---|---|

| `401 Unauthorized` | Check `AMROUTER_KEY`, ensure bearer token format: `Authorization: Bearer ${KEY}` |

| `503 Service Unavailable` | All fallback providers failed; check quota and provider configs in dashboard |

| `Model not found` | Run `curl ${AMROUTER_URL}/v1/models` to see available models |

| `CORS errors` | Add `Access-Control-Allow-Origin: *` in backend (or configure for specific origins) |

| Backend won't start | Check `.env` file, ensure `NODE_ENV` is set, check port conflicts |

| Embeddings dimension mismatch | Different models have different dimensions (OpenAI: 1536, Gemini: 768). Store model name with vectors. |

---

## Deployment

### Docker

```dockerfile

FROM node:20-alpine

WORKDIR /app

COPY . .

RUN npm install

RUN cd frontend && npm run build

CMD ["npm", "start"]

```

```bash

docker build -t amrouter .

docker run -p 3001:3001 --env-file .env amrouter

```

### Kubernetes

```yaml

apiVersion: apps/v1

kind: Deployment

metadata:

spec:

replicas: 3

selector:

matchLabels:

app: amrouter

template:

metadata:

labels:

app: amrouter

spec:

containers:

- name: amrouter

image: amrouter:latest

ports:

- containerPort: 3001

env:

- name: PORT

value: "3001"

- name: ADMIN_PASSWORD

valueFrom:

secretKeyRef:

key: admin-password

```

### Environment Variables (Production)

```bash

PORT=3001

NODE_ENV=production

REQUIRE_LOGIN=true

JWT_SECRET=<random-256-bit-hex>

ADMIN_PASSWORD=<strong-password>

CORS_ORIGIN=https://yourdomain.com

LOG_LEVEL=info

```

---

## Integration with Agent Skills

| Skill | Use Case |

|---|---|

| `deep-agents-core` | Register AMRouter endpoints as custom LLM tools |

| `research` | Parallelize queries across multiple models in AMRouter |

| `crewai-multi-agent` | Use AMRouter as shared LLM backend for multi-agent orchestration |

| `openrlhf-training` | Use AMRouter for inference during RLHF fine-tuning |

| `langgraph-human-in-the-loop` | Route LangGraph LLM calls through AMRouter combo chains |

---

## License

AMRouter is open-source. See [GitHub](https://github.com/ahwanulm/AMRouter) for license details.

---

**Built with:** Node.js + Express (backend), Vite + React (frontend), OpenAI-compatible API

**Last updated:** July 2026 — reflects AMRouter v2.0+