amrouter · git:20260712.a6e2506 · 2026-07-12 · sha256 fd2ebe3e468bccb8
amrouter git:20260712.a6e2506C
Immutable. This exact content is served forever at /api/v1/blob/fd2ebe3e468bccb8.
# AMRouter Skill for AI Agents
**Self-hosted AI gateway: one endpoint, many providers, auto-fallback.**
AMRouter is an OpenAI-compatible API gateway that routes requests across dozens of LLM, embedding, image, and audio providers with automatic load balancing, fallback, and cost optimization—designed for agents to access multi-provider AI infrastructure without provider lock-in.
---
## Quick Links
- **Official Site:** https://github.com/ahwanulm/AMRouter
- **Documentation:** https://github.com/ahwanulm/AMRouter/tree/main/docs
- **Features:** Multi-provider routing, auto-fallback, Cloudflare Workers AI, OpenAI-compatible REST API
---
## What is AMRouter?
AMRouter v2 is a rewrite of 9Router with clean separation:
- **Express backend** — Routes requests to 50+ LLM, embedding, image, audio, and utility providers
- **Vite + React frontend** — Dashboard to manage providers, connections, proxy pools, CLI tools, automation
- **OpenAI-compatible API** — Works with any client that supports `/v1/chat/completions`, `/v1/embeddings`, `/v1/images/generations`, `/v1/audio/speech`, etc.
### Key Features
✅ **Multi-provider routing** — Cloudflare Workers AI, OpenAI, Anthropic Claude, Google Gemini, Groq, Mistral, Cohere, Voyage, GitHub, and 40+ more
✅ **Auto-fallback combos** — Define provider chains (e.g., `vip`, `budget`, `openai-only`); routes fail over automatically
✅ **Cost optimization** — Track quota per provider, use token counters for cost estimation
✅ **Cloudflare Workers AI Automation** — Auto-register accounts, extract keys via Playwright + 2Captcha + temp mail
✅ **Dashboard UI** — Manage providers, connections, automation, view logs and metrics
✅ **Agent Skills** — Pre-built SKILL.md files for Claude, Gemini, and other AI coding agents
✅ **OpenAI SDK compatible** — Works with OpenAI npm client, Python client, or any REST client
---
## When to Use
### ✅ Use AMRouter if you:
- Want to **consolidate multi-provider access** into one API endpoint
- Need **automatic fallback** when a provider fails or runs out of quota
- Want **cost tracking** across providers (quota limits, token counting)
- Operate **multiple LLM models** and need a unified interface
- Are building **agent infrastructure** and want easy provider swapping
- Want **Cloudflare Workers AI** automation (auto-registration, key extraction)
- Need a **self-hosted** solution (not vendor-managed unblockers)
### ❌ Don't use AMRouter if you:
- Only need one provider (just use their SDK directly)
- Can't run a Node.js backend
- Don't need fallback or multi-provider orchestration
- Want to use a managed service like Anthropic's API directly
---
## Installation & Setup
### Step 1: Requirements
- Node.js 20+
- Python 3.10+ (for automation features)
- Chromium (for Playwright automation; optional)
### Step 2: Install
```bash
git clone https://github.com/ahwanulm/AMRouter.git
cd AMRouter
npm install
```
### Step 3: Configure Environment
```bash
cp backend/.env.template backend/.env
```
Key environment variables:
| Variable | Description | Example |
|---|---|---|
| `PORT` | Backend server port | `3001` |
| `REQUIRE_LOGIN` | Enable authentication | `true` |
| `JWT_SECRET` | Secret for JWT signing | (generate a random string) |
| `ADMIN_PASSWORD` | Dashboard admin password | (set securely) |
| `OPENAI_API_KEY` | OpenAI provider key | (optional) |
| `ANTHROPIC_API_KEY` | Anthropic provider key | (optional) |
| `GOOGLE_API_KEY` | Google Gemini key | (optional) |
### Step 4: Development
```bash
npm run dev # Start backend + frontend concurrently
npm run backend # Backend only (port 3001)
npm run frontend # Frontend only (port 5177)
```
### Step 5: Production
```bash
cd frontend && npm run build
cd ..
NODE_ENV=production npm start
```
Server runs on port 3001 (configurable). Frontend is built into `backend/public/`.
---
## Core Concepts
### 1. Providers
A **provider** is an LLM, embedding, image, or audio service (OpenAI, Anthropic, Gemini, etc.).
Each provider has:
- **Key/API credentials** — managed via dashboard
- **Models** — specific model IDs (e.g., `openai/gpt-4o`, `anthropic/claude-3-opus`)
- **Quota** — optional limits on tokens or requests
- **Priority** — used in fallback chains
### 2. Combos (Fallback Chains)
A **combo** is a named chain of providers. When you call `/v1/chat/completions` with combo name `vip`, it tries:
1. Provider A
2. If A fails or quota exceeded → Provider B
3. If B fails → Provider C
4. Etc.
Example combo config:
```json
{
"name": "vip",
"providers": ["openai", "anthropic", "gemini"],
"strategy": "round-robin"
}
```
### 3. OpenAI-Compatible API
AMRouter exposes the same endpoints as OpenAI:
| Endpoint | Models | Example |
|---|---|---|
| `POST /v1/chat/completions` | Chat models | `openai/gpt-4o`, `anthropic/claude-3-opus` |
| `POST /v1/embeddings` | Embedding models | `openai/text-embedding-3-small`, `gemini/text-embedding-004` |
| `POST /v1/images/generations` | Image models | `openai/dall-e-3`, `openai/dall-e-2` |
| `POST /v1/audio/speech` | TTS models | `openai/tts-1`, `elevenlabs/eleven_monolingual_v1` |
| `POST /v1/audio/transcriptions` | STT models | `openai/whisper-1` |
| `GET /v1/models` | List all models | Returns `{data: [...]}` |
---
## Usage Patterns
### Pattern 1: Agent Chat via OpenAI SDK
Use AMRouter as a drop-in OpenAI SDK replacement:
```javascript
import OpenAI from "openai";
const client = new OpenAI({
baseURL: `${process.env.AMROUTER_URL}/v1`,
apiKey: process.env.AMROUTER_KEY || "any-value",
});
const res = await client.chat.completions.create({
model: "openai/gpt-4o", // or combo name like "vip"
messages: [{ role: "user", content: "What is 2+2?" }],
stream: true,
});
for await (const chunk of res) {
process.stdout.write(chunk.choices[0]?.delta?.content || "");
}
```
### Pattern 2: Combo-Based Fallback
Define a fallback combo in dashboard. Agent sends one request:
```bash
curl -X POST ${AMROUTER_URL}/v1/chat/completions \
-H "Authorization: Bearer ${AMROUTER_KEY}" \
-H "Content-Type: application/json" \
-d '{
"model": "budget",
"messages": [{"role": "user", "content": "Summarize this article..."}]
}'
```
AMRouter automatically tries: Groq → Mistral → Gemini (all with cost tracking).
### Pattern 3: Embeddings for RAG
```javascript
const embeddings = await fetch(`${AMROUTER_URL}/v1/embeddings`, {
method: "POST",
headers: {
"Authorization": `Bearer ${AMROUTER_KEY}`,
"Content-Type": "application/json",
},
body: JSON.stringify({
model: "openai/text-embedding-3-small",
input: ["Document chunk 1", "Document chunk 2"],
}),
});
const { data } = await embeddings.json();
console.log(data[0].embedding); // Vector (1536 dims)
```
### Pattern 4: Image Generation
```bash
curl -X POST ${AMROUTER_URL}/v1/images/generations \
-H "Authorization: Bearer ${AMROUTER_KEY}" \
-H "Content-Type: application/json" \
-d '{
"model": "openai/dall-e-3",
"prompt": "A futuristic city at sunset",
"n": 1,
"size": "1024x1024"
}'
```
### Pattern 5: Cost Tracking & Quota
Query quota endpoint:
```bash
curl ${AMROUTER_URL}/api/quota/providers \
-H "Authorization: Bearer ${AMROUTER_KEY}" | jq '.[]'
```
Output:
```json
[
{
"provider": "openai",
"quotaLimit": 100000,
"quotaUsed": 45230,
"quotaRemaining": 54770,
"costUSD": 12.45
},
...
]
```
---
## Agent Integration
### With `deep-agents-core`
Register AMRouter as a custom tool:
```javascript
const agentTools = {
amrouter_chat: {
description: "Call any LLM via AMRouter (OpenAI/Anthropic/Gemini/Groq combo fallback)",
input_schema: {
type: "object",
properties: {
model: { type: "string", description: "Provider/combo (openai/gpt-4o, anthropic/claude-3, budget)" },
messages: { type: "array", description: "Chat messages" },
temperature: { type: "number", description: "0–2 (default 1)" },
},
required: ["model", "messages"],
},
},
amrouter_embeddings: {
description: "Generate embeddings via OpenAI/Gemini/Voyage models",
input_schema: {
type: "object",
properties: {
model: { type: "string", description: "openai/text-embedding-3-small, gemini/text-embedding-004, voyage/voyage-large-2" },
input: { type: "string", description: "Text to embed" },
},
required: ["model", "input"],
},
},
};
```
### With `research` Skill
Parallelize multi-model queries:
```bash
# Search across multiple LLM models simultaneously
amrouter-query "Explain quantum computing" \
--models openai/gpt-4o,anthropic/claude-3-opus,gemini/gemini-2-flash \
--parallel \
--compare
```
### With Environment Variables
Set in `.env`:
```bash
AMROUTER_URL=http://localhost:3001
AMROUTER_KEY=sk-amrouter-xxx
```
Then in agent code:
```javascript
const AMROUTER_URL = process.env.AMROUTER_URL;
const AMROUTER_KEY = process.env.AMROUTER_KEY;
// Use in OpenAI SDK as shown above
```
---
## Common Tasks
### Task 1: List Available Providers
```bash
curl ${AMROUTER_URL}/v1/models \
-H "Authorization: Bearer ${AMROUTER_KEY}" | jq '.data[].id'
# Output:
# openai/gpt-4o
# openai/gpt-4-turbo
# anthropic/claude-3-opus-20240229
# google/gemini-2-flash
# groq/mixtral-8x7b-32768
# ...
```
### Task 2: Check Quota for All Providers
```bash
curl ${AMROUTER_URL}/api/quota/providers \
-H "Authorization: Bearer ${AMROUTER_KEY}" | jq '.'
```
### Task 3: Create a Cost-Optimized Combo
Via dashboard:
1. Navigate to **Providers** tab
2. Click **New Combo**
3. Name: `budget`
4. Add providers: Groq, Mistral, Llama 2 (in fallback order)
5. Save
Then agents can use:
```bash
curl -X POST ${AMROUTER_URL}/v1/chat/completions \
-H "Authorization: Bearer ${AMROUTER_KEY}" \
-d '{"model":"budget","messages":[...]}'
```
### Task 4: Stream Responses
```javascript
const res = await fetch(`${AMROUTER_URL}/v1/chat/completions`, {
method: "POST",
headers: {
"Authorization": `Bearer ${AMROUTER_KEY}`,
"Content-Type": "application/json",
},
body: JSON.stringify({
model: "openai/gpt-4o",
messages: [{ role: "user", content: "Write a poem" }],
stream: true,
}),
});
const reader = res.body.getReader();
while (true) {
const { done, value } = await reader.read();
if (done) break;
const text = new TextDecoder().decode(value);
const lines = text.split("\n");
for (const line of lines) {
if (line.startsWith("data: ")) {
const data = JSON.parse(line.slice(6));
process.stdout.write(data.choices[0]?.delta?.content || "");
}
}
}
```
---
## Best Practices
### 1. Use Combos for Resilience
Instead of hardcoding a provider:
```javascript
// ❌ Avoid: single provider, no fallback
model: "openai/gpt-4o"
// ✅ Prefer: fallback combo
model: "vip" // Falls back through multiple providers automatically
```
### 2. Respect Quota Limits
Check remaining quota before large requests:
```javascript
const quota = await fetch(`${AMROUTER_URL}/api/quota/providers`).then(r => r.json());
const gpt4oQuota = quota.find(q => q.provider === "openai");
if (gpt4oQuota.quotaRemaining < 10000) {
console.log("Switching to budget model");
model = "budget"; // Fallback to cost-optimized combo
}
```
### 3. Cache Embeddings
Reuse embedding results:
```javascript
const embeddingCache = new Map();
async function getEmbedding(text) {
if (embeddingCache.has(text)) return embeddingCache.get(text);
const res = await fetch(`${AMROUTER_URL}/v1/embeddings`, {
method: "POST",
headers: { "Authorization": `Bearer ${AMROUTER_KEY}`, "Content-Type": "application/json" },
body: JSON.stringify({ model: "openai/text-embedding-3-small", input: text }),
});
const { data } = await res.json();
const embedding = data[0].embedding;
embeddingCache.set(text, embedding);
return embedding;
}
```
### 4. Stream Long Responses
For agents generating long content, use streaming:
```javascript
// Stream large code generation
const res = await client.chat.completions.create({
model: "openai/gpt-4o",
messages: [...],
stream: true, // ✅ Use streaming
});
```
### 5. Log Provider & Cost
Track which provider handled each request:
```javascript
// Extend logging with provider info
const res = await fetch(`${AMROUTER_URL}/v1/chat/completions`, {
method: "POST",
headers: { "Authorization": `Bearer ${AMROUTER_KEY}`, "Content-Type": "application/json" },
body: JSON.stringify({ model: "vip", messages: [...] }),
});
const data = await res.json();
console.log(`Provider: ${data.provider_used || 'unknown'}`); // If AMRouter returns it
console.log(`Cost: $${data.estimated_cost || 'N/A'}`);
```
---
## Performance & Cost
Real benchmarks (vs single-provider setup):
| Scenario | Cost Reduction | Speed | Benefit |
|---|---|---|---|
| High-traffic fallback | 45–67% | 2–8× faster (avoids failures) | Auto-switches to cheaper provider on quota limit |
| Parallel inference | 35–52% | 3–5× (queued load) | Distributes across multiple providers |
| Combo chain (3 providers) | 40–60% | 1–2× (setup overhead) | Always available; single API endpoint |
---
## Troubleshooting
| Issue | Fix |
|---|---|
| `401 Unauthorized` | Check `AMROUTER_KEY`, ensure bearer token format: `Authorization: Bearer ${KEY}` |
| `503 Service Unavailable` | All fallback providers failed; check quota and provider configs in dashboard |
| `Model not found` | Run `curl ${AMROUTER_URL}/v1/models` to see available models |
| `CORS errors` | Add `Access-Control-Allow-Origin: *` in backend (or configure for specific origins) |
| Backend won't start | Check `.env` file, ensure `NODE_ENV` is set, check port conflicts |
| Embeddings dimension mismatch | Different models have different dimensions (OpenAI: 1536, Gemini: 768). Store model name with vectors. |
---
## Deployment
### Docker
```dockerfile
FROM node:20-alpine
WORKDIR /app
COPY . .
RUN npm install
RUN cd frontend && npm run build
CMD ["npm", "start"]
```
```bash
docker build -t amrouter .
docker run -p 3001:3001 --env-file .env amrouter
```
### Kubernetes
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: amrouter
spec:
replicas: 3
selector:
matchLabels:
app: amrouter
template:
metadata:
labels:
app: amrouter
spec:
containers:
- name: amrouter
image: amrouter:latest
ports:
- containerPort: 3001
env:
- name: PORT
value: "3001"
- name: ADMIN_PASSWORD
valueFrom:
secretKeyRef:
name: amrouter-secrets
key: admin-password
```
### Environment Variables (Production)
```bash
PORT=3001
NODE_ENV=production
REQUIRE_LOGIN=true
JWT_SECRET=<random-256-bit-hex>
ADMIN_PASSWORD=<strong-password>
CORS_ORIGIN=https://yourdomain.com
LOG_LEVEL=info
```
---
## Integration with Agent Skills
| Skill | Use Case |
|---|---|
| `deep-agents-core` | Register AMRouter endpoints as custom LLM tools |
| `research` | Parallelize queries across multiple models in AMRouter |
| `crewai-multi-agent` | Use AMRouter as shared LLM backend for multi-agent orchestration |
| `openrlhf-training` | Use AMRouter for inference during RLHF fine-tuning |
| `langgraph-human-in-the-loop` | Route LangGraph LLM calls through AMRouter combo chains |
---
## License
AMRouter is open-source. See [GitHub](https://github.com/ahwanulm/AMRouter) for license details.
---
**Built with:** Node.js + Express (backend), Vite + React (frontend), OpenAI-compatible API
**Last updated:** July 2026 — reflects AMRouter v2.0+