---
name: hor-multi-llm-provider
description: >
  Support multiple LLM providers (Anthropic Claude / OpenAI / Google Gemini) behind one
  abstraction in a renchan backend: an abstract model processor, one provider base per vendor, one
  concrete processor per model, and a loader that picks one by model name at runtime, with model
  and provider metadata in the DB and env, not in code. Use whenever the user asks to add an LLM
  model or provider, change how a model is selected, normalize provider responses, or wire
  function calls and file upload.
---

# Multi-LLM Provider (Claude / OpenAI / Gemini behind one abstraction)

This backend talks to three LLM vendors through **one processor abstraction** so callers (agents,
resolvers, jobs) never branch on the provider. A caller resolves a **model name** to a processor and calls
`processor.sendRequestToAi(...)`; everything provider-specific (SDK calls, function-call shape, file
upload, response parsing) is hidden behind the processor and its capsule.

Which **model** answers and which **access route** carries the request are two separate choices. The
model is a vendor's model (Claude Sonnet, Gemini Flash). The route is how the request reaches it and how
it authenticates: the vendor's own API with an API key, or a cloud platform that hosts the model.

## Grand principle: one base + one subclass per model; provider differences live in a provider base, the model is chosen from the DB

The abstraction is a three-level strategy: `BaseAiModelProcessor` (the contract) → one **provider base**
per vendor (Claude/OpenAI/Gemini plumbing) → one **concrete processor per model** whose only distinct
member is `#get:aiModel`. A loader auto-discovers the concrete processors and picks one by model-name
string at runtime.

- **Why a per-model class whose only identity is `#get:aiModel`**: adding a model must be additive
  (Open–Closed) — drop one file whose `#get:aiModel` returns the new name, and the loader finds it. The
  concrete class holds *no* target model id or token limit; those come from the DB `AiModel` /
  `AiModelCapability` rows, so a model's real endpoint and limits are data, not code.
- **Why a provider base between them**: the three vendors differ only in a few mechanics —
  function-call event shape (`tool_use` vs `function_call` vs `functionCall`), how tool results are fed
  back (`messages` vs `input` vs `contents`), and file upload (Claude file id / OpenAI file id / Gemini
  file uri). Concentrating those in one provider base per vendor keeps every concrete model processor
  identical except for its name.
- **Why normalize responses through `AiModelResponse`**: callers must read `#hasError()`,
  `#extractFunctionCalls()`, `#extractContentText()` the same way regardless of vendor. Each provider's
  capsule exposes the same method surface; `AiModelResponse` is the thin uniform wrapper over it.
- **Why choose the model from the DB**: which model an agent uses is per-agent config
  (`AiAgentDefaultModel → AiModel.name`), overridable per call — never hard-coded. Swapping a model is a
  data change.
- **Why the access route is separate from the model**: one model is often reachable several ways, and each
  way differs in the client, the credentials, and the vendor model id (the same Claude model has a
  different id on the vendor's API than on a cloud platform). Putting the route in the provider base or
  the concrete processor would need one class per model per route. A route class owns the client and the
  authentication instead, and the model's DB rows say which routes reach it.

## Layers and files

```
app/tools/
  BaseAiModelProcessor.js                     abstract contract (get aiModel, sendRequestToAi, ...)
  BaseAiModelProviderProcessor/
    BaseClaudeAIProcessor.js                  Claude function-call loop + Files API upload
    BaseOpenAIProcessor.js                    OpenAI function_call + file upload (with expiry)
    BaseGeminiAIProcessor.js                  Gemini functionCall + file uri upload
  AiModelProcessor/
    ClaudeSonnet4_5AiModelProcessor.js        concrete: get aiModel + sendRequestToAi
    OpenAiGPT_5_AiModelProcessor.js           (~21 concrete model files, one per model)
    Gemini2_5FlashAiModelProcessor.js
    ...
  BulkAiModelProcessorsLoader.js              auto-discovery + getProcessor(aiModel)
  AiPayloadGenerator/
    BaseAiMessagePayloadGenerator.js          shared history trimming / token budget
    ClaudeMessagePayloadGenerator.js          provider payload shaping (tools, toolChoice, files)
    OpenAiMessagePayloadGenerator.js
    GeminiMessagePayloadGenerator.js
  AiModelResponse.js                          normalized response wrapper
  AiAccessRoute/
    BaseAiAccessRoute.js                      abstract contract (provider, route name, client, auth)
    ClaudeDirectAiAccessRoute.js              concrete: Claude through the vendor API key
    GeminiDirectAiAccessRoute.js              concrete: Gemini through the vendor API key
    OpenAiDirectAiAccessRoute.js              concrete: OpenAI through the vendor API key
    ClaudeVertexAiAccessRoute.js              concrete: Claude on Vertex AI, service account
    GeminiVertexAiAccessRoute.js              concrete: Gemini on Vertex AI, service account
    ClaudeBedrockAiAccessRoute.js             concrete: Claude on Amazon Bedrock, AWS credentials
  BulkAiAccessRoutesLoader.js                 auto-discovery + getRoute({ aiProviderId, accessRouteName })
app/claudeClient/ | app/geminiClient/ | app/openAiClient/   low-level Launcher/Payload/Capsule/Client
```

## 1. The contract — `BaseAiModelProcessor`

Abstract members a concrete processor must satisfy:

- `#get:aiModel` (abstract) — the model-name string the loader matches on (e.g. `'sonnet-4-5'`).
- `#sendRequestToAi({ ... })` (abstract) — non-streaming request; returns an `AiModelResponse`.
- `#sendStreamRequestToAi({ ..., onText, onComplete })` (abstract) — streaming variant.
- `#prepareAttachedFiles({ fileUrls })` (concrete default: returns `fileUrls` unchanged) — overridden per
  provider to upload files and attach provider file ids/uris.
- `#findAiModelByName({ aiModelName })` (concrete) —
  `AiModel.findOne({ where: { name }, include: [AiModelCapability, AiModelAccessRoute] })`; this is how a
  processor gets its DB-backed `maxOutputToken` and its routes.
- `#findAccessRoute({ aiModel })` (concrete), picks the model's default active `AiModelAccessRoute` row and
  returns `{ accessRoute, targetModelName }`: the route instance from the loader and the vendor model id
  on that route (see section 4).

The request param bag is uniform across providers:

```js
async sendRequestToAi ({
  aiAgent,
  instruction,
  documents,
  fileUrls,
  historyMessages = [],
  tools = [],
  toolChoices = [],
  isAutoHandleFunctionCall = true,
  extraToolOptions = {},
  shouldPrefetchDynamicDocumentsForBackground = false,
}) {
  // ...
}
```

## 2. A concrete model processor — only `#get:aiModel` is unique

Everything else (build the instruction, prepare files, build the payload, send, handle one round of
function calls, wrap the response) is the same standard flow inherited/composed from the provider base.

```js
import BaseClaudeAIProcessor from '../BaseAiModelProviderProcessor/BaseClaudeAIProcessor.js'
import AiModelResponse from '../AiModelResponse.js'

import CONSTANT_HASH from '../../../constants/aiConstants.cjs'

const {
  AI_MODEL,
} = CONSTANT_HASH

export default class ClaudeSonnet4_5AiModelProcessor extends BaseClaudeAIProcessor {
  static create () {
    return new this()
  }

  get aiModel () {
    return AI_MODEL.CLAUDE_SONNET_4_5.NAME
  }

  async sendRequestToAi ({
    aiAgent,
    instruction,
    documents,
    fileUrls,
    historyMessages = [],
    tools = [],
    toolChoices = [],
    isAutoHandleFunctionCall = true,
    extraToolOptions = {},
    shouldPrefetchDynamicDocumentsForBackground = false,
  }) {
    const aiModel = await this.findAiModelByName({
      aiModelName: this.aiModel,
    })

    const documentInstructionComposer = await this.createDocumentInstructionComposer({
      // ...splits documents, wires the agent's default instruction (see `hor-ai-prompt-document-store` skill)
    })

    const preparedFileUrls = await this.prepareAttachedFiles({
      fileUrls,
    })

    const payloadGenerator = this.createClaudeMessagePayloadGenerator({
      aiAgent,
      aiModel,
      message: documentInstructionComposer.generateComposedInstruction(),
      fileUrls: preparedFileUrls,
      historyMessages,
      tools,
      toolChoices,
    })

    const payload = payloadGenerator.generateClaudeMessagePayload()
    const fetcher = this.createSendMessageToClaudeFetcher()
    const capsule = await fetcher.launchRequest(payload)

    if (capsule.hasError()) {
      return AiModelResponse.create({
        aiResponseCapsule: capsule,
      })
    }

    if (!capsule.hasFunctionCallEvent()) {
      return AiModelResponse.create({
        aiResponseCapsule: capsule,
      })
    }

    if (!isAutoHandleFunctionCall) {
      return AiModelResponse.create({
        aiResponseCapsule: capsule,
      })
    }

    const followUpPayload = await this.handleFunctionCalls({
      payloadParams: payload.params,
      aiResponses: capsule.extractContent(),
      extraToolOptions,
    })

    const followUpCapsule = await fetcher.launchRequest(followUpPayload)

    return AiModelResponse.create({
      aiResponseCapsule: followUpCapsule,
    })
  }
}
```

- **`targetModelName` and `maxOutputToken` come from the DB row**, not the class. `#get:aiModel` returns
  the app-facing name (`AI_MODEL.*.NAME`); `targetModelName` is the vendor model id on the chosen access
  route (e.g. `'claude-sonnet-4-5-20250929'` on the vendor API, see section 4).
- **`isAutoHandleFunctionCall: false`** short-circuits after the first response and returns the raw
  function call — this is exactly what the forced-single-tool actions in the **`hor-ai-agent-structure` skill**
  rely on.
- The flow handles **one** round of function calls (one follow-up request), not an open-ended tool loop.

## 3. Runtime selection — `BulkAiModelProcessorsLoader`

The loader auto-discovers every class under `app/tools/AiModelProcessor/` with renchan's
`DeepBulkClassLoader`, instantiates each via `.create()`, and picks one by matching `#get:aiModel`:

```js
import {
  DeepBulkClassLoader,
} from '@openreachtech/renchan'

// ...
static async loadProcessorClasses (path) {
  return DeepBulkClassLoader.create({
    poolPath: path,
  })
    .loadClasses()
}

getProcessor (aiModel) {
  return this.processors.find(processor => processor.aiModel === aiModel)
}
```

The model **name** passed to `#getProcessor()` comes from the DB — the agent's default model, overridable
per call:

```js
// explicit id wins, else the agent's default model
const targetAiModelId = aiModelId
  ?? aiAgentDefaultModel?.AiModelId
  ?? null

const aiModel = await AiModel.findByPk(targetAiModelId)
const processor = this.bulkAiModelProcessorsLoader.getProcessor(aiModel.name)
```

or directly via the agent's default-model chain
`aiAgent.AiAgentDefaultModel.AiModel.name → getProcessor(...)`.

> Do not confuse this with `BulkAiToolProcessorsLoader`: the provider bases use a *tool*-processor loader
> (`getProcessor(toolName)`) to run function calls — a different loader over `app/aiTools/`.

## 4. Access routes

An access route is a class under `app/tools/AiAccessRoute/`, one per provider and route pair. It holds
the two things that change with the route and nothing else: the low-level client and the credentials
that client authenticates with. Prompting, payload shaping, function-call handling and response
parsing stay in the provider base, so they are written once per vendor and never per route.

`BaseAiAccessRoute` contract:

- `#get:aiProviderId` (abstract), the `AI_PROVIDER.*.ID` this route serves. A route never changes the
  provider: Claude reached through a cloud platform is still the Anthropic provider.
- `#get:accessRouteName` (abstract), the `AI_ACCESS_ROUTE.*.NAME` the loader matches on.
- `#get:supportsFileUpload` (abstract), whether `#prepareAttachedFiles()` may upload through this route.
- `#createClient()` (abstract), returns the client the Fetcher already calls
  (`this.client.sendMessageToClaude({ ...p })`), built from env, never from a literal. For the vendor API
  this is the construction `Base<Provider>ApiRequestLauncher` did when it read the key itself; that read
  moves here.
- `#createHttpRequestClient()` (concrete), the HTTP client the vendor-API clients wrap.

```js
import environment from '../../globals/env.js'
import ClaudeApiClient from '../../claudeClient/ClaudeApiClient.js'
import BaseAiAccessRoute from './BaseAiAccessRoute.js'

import CONSTANT_HASH from '../../../constants/aiConstants.cjs'

const {
  AI_PROVIDER,
  AI_ACCESS_ROUTE,
} = CONSTANT_HASH

export default class ClaudeDirectAiAccessRoute extends BaseAiAccessRoute {
  static create () {
    return new this()
  }

  get aiProviderId () {
    return AI_PROVIDER.ANTHROPIC.ID
  }

  get accessRouteName () {
    return AI_ACCESS_ROUTE.DIRECT.NAME
  }

  get supportsFileUpload () {
    return true
  }

  createClient () {
    return ClaudeApiClient.create({
      token: environment.CLAUDE_API_TOKEN,
      httpRequestClient: this.createHttpRequestClient(),
    })
  }
}
```

**Choosing the route is a data lookup, like choosing the model.** A model's reachable routes are
`AiModelAccessRoute` rows (see section 6). The processor resolves the route after it resolves the model:

```js
const aiModel = await this.findAiModelByName({
  aiModelName: this.aiModel,
})

const {
  accessRoute,
  targetModelName,
} = await this.findAccessRoute({
  aiModel,
})

const fetcher = this.createSendMessageToClaudeFetcher({
  client: accessRoute.createClient(),
})
```

- **The vendor model id comes from the route row**, not from `AiModel.targetModelName`: the payload
  generator receives `targetModelName` from `#findAccessRoute()`. A model with no route row falls back to the
  `direct` route and `AiModel.targetModelName`, so models that exist before routes keep working.
- **A route that cannot upload files** (`supportsFileUpload` is `false`) makes `#prepareAttachedFiles()`
  return the files unchanged, so the provider id/uri stays unset and the payload generator takes its
  non-upload source (see `provider-internals.md`).
- **Callers still never see the route.** `#sendRequestToAi()` and `AiModelResponse` are identical for
  every route, because each route's client returns the same response shape the provider capsule parses.

See [access-routes.md](./references/access-routes.md) for the loader, the route rows, and what each
route needs.

## 5. Adding a model, a provider, or a route

- **New model of an existing provider**: add one file under `app/tools/AiModelProcessor/` extending that
  provider base, with `#get:aiModel` returning the new `AI_MODEL.*.NAME`; add the `AiModel` +
  `AiModelCapability` rows (and `AiModelToolAssignment` rows) via a seeder; add the `AI_MODEL` entry in
  `constants/aiConstants.cjs`. No caller changes — the loader finds it.
- **New provider**: add a provider base under `app/tools/BaseAiModelProviderProcessor/` implementing the
  function-call handling + `#prepareAttachedFiles()` for that vendor, a payload generator under
  `AiPayloadGenerator/`, a capsule that exposes the standard method surface, the low-level
  Launcher/Payload/Capsule/Client under `app/<provider>Client/`, and an `AI_PROVIDER` entry. Then a
  `direct` access route for it and per-model processors as above.
- **New route for an existing provider**: add one file under `app/tools/AiAccessRoute/` extending
  `BaseAiAccessRoute`, an `AI_ACCESS_ROUTE` entry, the env vars it reads, and one `AiModelAccessRoute` row
  per model that route reaches. No processor or caller changes.

See [provider-internals.md](./references/provider-internals.md) for the per-vendor function-call and
file-upload differences, the payload generators, `AiModelResponse`, and the low-level client pattern.

## 6. Config and secrets

- **`AiProvider`** (`ai_providers`): `name`. `AI_PROVIDER = { ANTHROPIC {ID:1}, GOOGLE {ID:2}, OPENAI {ID:3} }`.
- **`AiModel`** (`ai_models`): `AiProviderId`, `name` (app key), `targetModelName` (vendor id on the `direct`
  route), `isDefault`, `isActive`, `displayOrder`.
- **`AiModelCapability`** (`ai_model_capabilities`): `contextWindowToken`, `maxOutputToken` (→ payload
  `maxTokens`).
- **`AiModelAccessRoute`** (`ai_model_access_routes`): `AiModelId`, `accessRouteName` (an
  `AI_ACCESS_ROUTE.*.NAME`), `targetModelName` (the vendor id on that route), `isDefault`, `isActive`.
- **`AiModelToolAssignment`** (`ai_model_tool_assignments`): which tools a model may use (see the
  `hor-ai-prompt-document-store` skill for `AiTool`).
- **API keys / base URLs** come from `@openreachtech/renchan-env` via `app/globals/env.js` (e.g.
  `CLAUDE_API_TOKEN`, `CLAUDE_API_BASE_URL`, and the analogous OpenAI/Gemini vars) — never hard-code a key.
  Only the route that uses a credential reads it, so each route's env vars are listed in
  [access-routes.md](./references/access-routes.md).
- **Vertex AI uses a service account, not a key**: Application Default Credentials supply it, and only the
  project id and location come from env. Keep a service-account key file out of the repository and the DB.
- **Bedrock uses AWS credentials**: the AWS default credential chain supplies them (prefer a role attached to
  the runtime), and only the region comes from env. Never set `AWS_BEARER_TOKEN_BEDROCK` where IAM
  credentials should sign.

`AI_ACCESS_ROUTE` names the routes: `DIRECT { NAME: 'direct' }`, `VERTEX_AI { NAME: 'vertex-ai' }` and
`BEDROCK { NAME: 'bedrock' }`.

Model/provider/route constants live in `constants/aiConstants.cjs` (see the **`hor-constant-definition` skill** for the
two-file `.cjs` master + ESM bridge rule).

## Cross-cutting rules

- **Callers never branch on the provider** — resolve a processor by name and use the uniform
  `#sendRequestToAi()` / `AiModelResponse` surface.
- **A route owns the client and the credentials, and a processor never reads a key.** Reading a
  credential outside the route that uses it spreads one secret across several classes.
- **A concrete model processor adds no logic beyond `#get:aiModel`** — put shared behavior on the provider
  base, per-vendor behavior on the provider base too, and model specifics in the DB.
- **Return `null` / a normalized error response, not `undefined`**, on failure; check `#hasError()` before
  reading content.
- One class per file, one `export default`, `static create()` factory; getters return a constant.

## Detail files

- [access-routes.md](./references/access-routes.md), the route contract in full, `BulkAiAccessRoutesLoader`,
  the `AiModelAccessRoute` rows, and the env vars and authentication each route needs.
- [provider-internals.md](./references/provider-internals.md) — the three provider bases' function-call
  feedback shapes and file-upload models (Claude/OpenAI/Gemini), the payload generators
  (tools/toolChoice/history trimming/file source), the `AiModelResponse` + capsule method surface, and the
  low-level Launcher/Payload/Capsule/Client pattern with where API keys are read.