git:20261016.81d4445 to git:20261016.1baf4e6

---

name: hor-multi-llm-provider

description: >

---

# Multi-LLM Provider (Claude / OpenAI / Gemini behind one abstraction)

This backend talks to three LLM vendors through **one processor abstraction** so callers (agents,

resolvers, jobs) never branch on the provider. A caller resolves a **model name** to a processor and calls

`processor.sendRequestToAi(...)`; everything provider-specific (SDK calls, function-call shape, file

upload, response parsing) is hidden behind the processor and its capsule.

Which **model** answers and which **access route** carries the request are two separate choices. The

model is a vendor's model (Claude Sonnet, Gemini Flash). The route is how the request reaches it and how

it authenticates: the vendor's own API with an API key, or a cloud platform that hosts the model.

## Grand principle: one base + one subclass per model; provider differences live in a provider base, the model is chosen from the DB

The abstraction is a three-level strategy: `BaseAiModelProcessor` (the contract) → one **provider base**

per vendor (Claude/OpenAI/Gemini plumbing) → one **concrete processor per model** whose only distinct

member is `#get:aiModel`. A loader auto-discovers the concrete processors and picks one by model-name

string at runtime.

- **Why a per-model class whose only identity is `#get:aiModel`**: adding a model must be additive

(Open–Closed) — drop one file whose `#get:aiModel` returns the new name, and the loader finds it. The

concrete class holds *no* target model id or token limit; those come from the DB `AiModel` /

`AiModelCapability` rows, so a model's real endpoint and limits are data, not code.

- **Why a provider base between them**: the three vendors differ only in a few mechanics —

function-call event shape (`tool_use` vs `function_call` vs `functionCall`), how tool results are fed

back (`messages` vs `input` vs `contents`), and file upload (Claude file id / OpenAI file id / Gemini

file uri). Concentrating those in one provider base per vendor keeps every concrete model processor

identical except for its name.

- **Why normalize responses through `AiModelResponse`**: callers must read `#hasError()`,

`#extractFunctionCalls()`, `#extractContentText()` the same way regardless of vendor. Each provider's

capsule exposes the same method surface; `AiModelResponse` is the thin uniform wrapper over it.

- **Why choose the model from the DB**: which model an agent uses is per-agent config

(`AiAgentDefaultModel → AiModel.name`), overridable per call — never hard-coded. Swapping a model is a

data change.

- **Why the access route is separate from the model**: one model is often reachable several ways, and each

way differs in the client, the credentials, and the vendor model id (the same Claude model has a

different id on the vendor's API than on a cloud platform). Putting the route in the provider base or

the concrete processor would need one class per model per route. A route class owns the client and the

authentication instead, and the model's DB rows say which routes reach it.

## Layers and files

```

app/tools/

BaseAiModelProcessor.js abstract contract (get aiModel, sendRequestToAi, ...)

BaseAiModelProviderProcessor/

BaseClaudeAIProcessor.js Claude function-call loop + Files API upload

BaseOpenAIProcessor.js OpenAI function_call + file upload (with expiry)

BaseGeminiAIProcessor.js Gemini functionCall + file uri upload

AiModelProcessor/

ClaudeSonnet4_5AiModelProcessor.js concrete: get aiModel + sendRequestToAi

OpenAiGPT_5_AiModelProcessor.js (~21 concrete model files, one per model)

Gemini2_5FlashAiModelProcessor.js

...

BulkAiModelProcessorsLoader.js auto-discovery + getProcessor(aiModel)

AiPayloadGenerator/

BaseAiMessagePayloadGenerator.js shared history trimming / token budget

ClaudeMessagePayloadGenerator.js provider payload shaping (tools, toolChoice, files)

OpenAiMessagePayloadGenerator.js

GeminiMessagePayloadGenerator.js

AiModelResponse.js normalized response wrapper

AiAccessRoute/

ClaudeDirectAiAccessRoute.js concrete: Claude through the vendor API key

GeminiDirectAiAccessRoute.js concrete: Gemini through the vendor API key

OpenAiDirectAiAccessRoute.js concrete: OpenAI through the vendor API key

ClaudeVertexAiAccessRoute.js concrete: Claude on Vertex AI, service account

GeminiVertexAiAccessRoute.js concrete: Gemini on Vertex AI, service account

ClaudeBedrockAiAccessRoute.js concrete: Claude on Amazon Bedrock, AWS credentials

BulkAiAccessRoutesLoader.js auto-discovery + getRoute({ aiProviderId, accessRouteName })

app/claudeClient/ | app/geminiClient/ | app/openAiClient/ low-level Launcher/Payload/Capsule/Client

```

## 1. The contract — `BaseAiModelProcessor`

Abstract members a concrete processor must satisfy:

- `#get:aiModel` (abstract) — the model-name string the loader matches on (e.g. `'sonnet-4-5'`).

- `#sendRequestToAi({ ... })` (abstract) — non-streaming request; returns an `AiModelResponse`.

- `#sendStreamRequestToAi({ ..., onText, onComplete })` (abstract) — streaming variant.

- `#prepareAttachedFiles({ fileUrls })` (concrete default: returns `fileUrls` unchanged) — overridden per

provider to upload files and attach provider file ids/uris.

- `#findAiModelByName({ aiModelName })` (concrete) —

`AiModel.findOne({ where: { name }, include: [AiModelCapability, AiModelAccessRoute] })`; this is how a

processor gets its DB-backed `maxOutputToken` and its routes.

- `#findAccessRoute({ aiModel })` (concrete), picks the model's default active `AiModelAccessRoute` row and

returns `{ accessRoute, targetModelName }`: the route instance from the loader and the vendor model id

on that route (see section 4).

The request param bag is uniform across providers:

```js

async sendRequestToAi ({

aiAgent,

instruction,

documents,

fileUrls,

historyMessages = [],

tools = [],

toolChoices = [],

isAutoHandleFunctionCall = true,

extraToolOptions = {},

shouldPrefetchDynamicDocumentsForBackground = false,

}) {

// ...

}

```

## 2. A concrete model processor — only `#get:aiModel` is unique

Everything else (build the instruction, prepare files, build the payload, send, handle one round of

function calls, wrap the response) is the same standard flow inherited/composed from the provider base.

```js

import BaseClaudeAIProcessor from '../BaseAiModelProviderProcessor/BaseClaudeAIProcessor.js'

import AiModelResponse from '../AiModelResponse.js'

import CONSTANT_HASH from '../../../constants/aiConstants.cjs'

const {

AI_MODEL,

} = CONSTANT_HASH

export default class ClaudeSonnet4_5AiModelProcessor extends BaseClaudeAIProcessor {

static create () {

return new this()

}

get aiModel () {

return AI_MODEL.CLAUDE_SONNET_4_5.NAME

}

async sendRequestToAi ({

aiAgent,

instruction,

documents,

fileUrls,

historyMessages = [],

tools = [],

toolChoices = [],

isAutoHandleFunctionCall = true,

extraToolOptions = {},

shouldPrefetchDynamicDocumentsForBackground = false,

}) {

const aiModel = await this.findAiModelByName({

aiModelName: this.aiModel,

})

const documentInstructionComposer = await this.createDocumentInstructionComposer({

// ...splits documents, wires the agent's default instruction (see `hor-ai-prompt-document-store` skill)

})

const preparedFileUrls = await this.prepareAttachedFiles({

fileUrls,

})

const payloadGenerator = this.createClaudeMessagePayloadGenerator({

aiAgent,

aiModel,

message: documentInstructionComposer.generateComposedInstruction(),

fileUrls: preparedFileUrls,

historyMessages,

tools,

toolChoices,

})

const payload = payloadGenerator.generateClaudeMessagePayload()

const capsule = await fetcher.launchRequest(payload)

if (capsule.hasError()) {

return AiModelResponse.create({

aiResponseCapsule: capsule,

})

}

if (!capsule.hasFunctionCallEvent()) {

return AiModelResponse.create({

aiResponseCapsule: capsule,

})

}

if (!isAutoHandleFunctionCall) {

return AiModelResponse.create({

aiResponseCapsule: capsule,

})

}

const followUpPayload = await this.handleFunctionCalls({

payloadParams: payload.params,

aiResponses: capsule.extractContent(),

extraToolOptions,

})

const followUpCapsule = await fetcher.launchRequest(followUpPayload)

return AiModelResponse.create({

aiResponseCapsule: followUpCapsule,

})

}

}

```

- **`targetModelName` and `maxOutputToken` come from the DB row**, not the class. `#get:aiModel` returns

the app-facing name (`AI_MODEL.*.NAME`); `targetModelName` is the vendor model id on the chosen access

route (e.g. `'claude-sonnet-4-5-20250929'` on the vendor API, see section 4).

- **`isAutoHandleFunctionCall: false`** short-circuits after the first response and returns the raw

function call — this is exactly what the forced-single-tool actions in the **`hor-ai-agent-structure` skill**

rely on.

- The flow handles **one** round of function calls (one follow-up request), not an open-ended tool loop.

## 3. Runtime selection — `BulkAiModelProcessorsLoader`

The loader auto-discovers every class under `app/tools/AiModelProcessor/` with renchan's

`DeepBulkClassLoader`, instantiates each via `.create()`, and picks one by matching `#get:aiModel`:

```js

import {

DeepBulkClassLoader,

} from '@openreachtech/renchan'

// ...

static async loadProcessorClasses (path) {

return DeepBulkClassLoader.create({

poolPath: path,

})

.loadClasses()

}

getProcessor (aiModel) {

return this.processors.find(processor => processor.aiModel === aiModel)

}

```

The model **name** passed to `#getProcessor()` comes from the DB — the agent's default model, overridable

per call:

```js

// explicit id wins, else the agent's default model

const targetAiModelId = aiModelId

?? aiAgentDefaultModel?.AiModelId

?? null

const aiModel = await AiModel.findByPk(targetAiModelId)

const processor = this.bulkAiModelProcessorsLoader.getProcessor(aiModel.name)

```

or directly via the agent's default-model chain

`aiAgent.AiAgentDefaultModel.AiModel.name → getProcessor(...)`.

> Do not confuse this with `BulkAiToolProcessorsLoader`: the provider bases use a *tool*-processor loader

> (`getProcessor(toolName)`) to run function calls — a different loader over `app/aiTools/`.

## 4. Access routes

An access route is a class under `app/tools/AiAccessRoute/`, one per provider and route pair. It holds

the two things that change with the route and nothing else: the low-level client and the credentials

that client authenticates with. Prompting, payload shaping, function-call handling and response

parsing stay in the provider base, so they are written once per vendor and never per route.

`BaseAiAccessRoute` contract:

- `#get:aiProviderId` (abstract), the `AI_PROVIDER.*.ID` this route serves. A route never changes the

provider: Claude reached through a cloud platform is still the Anthropic provider.

- `#get:accessRouteName` (abstract), the `AI_ACCESS_ROUTE.*.NAME` the loader matches on.

- `#get:supportsFileUpload` (abstract), whether `#prepareAttachedFiles()` may upload through this route.

- `#createClient()` (abstract), returns the client the Fetcher already calls

(`this.client.sendMessageToClaude({ ...p })`), built from env, never from a literal. For the vendor API

this is the construction `Base<Provider>ApiRequestLauncher` did when it read the key itself; that read

moves here.

- `#createHttpRequestClient()` (concrete), the HTTP client the vendor-API clients wrap.

```js

import environment from '../../globals/env.js'

import ClaudeApiClient from '../../claudeClient/ClaudeApiClient.js'

import CONSTANT_HASH from '../../../constants/aiConstants.cjs'

const {

AI_PROVIDER,

AI_ACCESS_ROUTE,

} = CONSTANT_HASH

export default class ClaudeDirectAiAccessRoute extends BaseAiAccessRoute {

static create () {

return new this()

}

get aiProviderId () {

return AI_PROVIDER.ANTHROPIC.ID

}

get accessRouteName () {

return AI_ACCESS_ROUTE.DIRECT.NAME

}

get supportsFileUpload () {

return true

}

createClient () {

return ClaudeApiClient.create({

token: environment.CLAUDE_API_TOKEN,

httpRequestClient: this.createHttpRequestClient(),

})

}

}

```

**Choosing the route is a data lookup, like choosing the model.** A model's reachable routes are

`AiModelAccessRoute` rows (see section 6). The processor resolves the route after it resolves the model:

```js

const aiModel = await this.findAiModelByName({

aiModelName: this.aiModel,

})

const {

accessRoute,

targetModelName,

} = await this.findAccessRoute({

aiModel,

})

const fetcher = this.createSendMessageToClaudeFetcher({

client: accessRoute.createClient(),

})

```

- **The vendor model id comes from the route row**, not from `AiModel.targetModelName`: the payload

generator receives `targetModelName` from `#findAccessRoute()`. A model with no route row falls back to the

`direct` route and `AiModel.targetModelName`, so models that exist before routes keep working.

- **A route that cannot upload files** (`supportsFileUpload` is `false`) makes `#prepareAttachedFiles()`

return the files unchanged, so the provider id/uri stays unset and the payload generator takes its

non-upload source (see `provider-internals.md`).

- **Callers still never see the route.** `#sendRequestToAi()` and `AiModelResponse` are identical for

every route, because each route's client returns the same response shape the provider capsule parses.

See [access-routes.md](./references/access-routes.md) for the loader, the route rows, and what each

route needs.

## 5. Adding a model, a provider, or a route

- **New model of an existing provider**: add one file under `app/tools/AiModelProcessor/` extending that

provider base, with `#get:aiModel` returning the new `AI_MODEL.*.NAME`; add the `AiModel` +

`AiModelCapability` rows (and `AiModelToolAssignment` rows) via a seeder; add the `AI_MODEL` entry in

`constants/aiConstants.cjs`. No caller changes — the loader finds it.

- **New provider**: add a provider base under `app/tools/BaseAiModelProviderProcessor/` implementing the

function-call handling + `#prepareAttachedFiles()` for that vendor, a payload generator under

`AiPayloadGenerator/`, a capsule that exposes the standard method surface, the low-level

Launcher/Payload/Capsule/Client under `app/<provider>Client/`, and an `AI_PROVIDER` entry. Then a

`direct` access route for it and per-model processors as above.

- **New route for an existing provider**: add one file under `app/tools/AiAccessRoute/` extending

`BaseAiAccessRoute`, an `AI_ACCESS_ROUTE` entry, the env vars it reads, and one `AiModelAccessRoute` row

per model that route reaches. No processor or caller changes.

See [provider-internals.md](./references/provider-internals.md) for the per-vendor function-call and

file-upload differences, the payload generators, `AiModelResponse`, and the low-level client pattern.

## 6. Config and secrets

- **`AiProvider`** (`ai_providers`): `name`. `AI_PROVIDER = { ANTHROPIC {ID:1}, GOOGLE {ID:2}, OPENAI {ID:3} }`.

- **`AiModel`** (`ai_models`): `AiProviderId`, `name` (app key), `targetModelName` (vendor id on the `direct`

route), `isDefault`, `isActive`, `displayOrder`.

- **`AiModelCapability`** (`ai_model_capabilities`): `contextWindowToken`, `maxOutputToken` (→ payload

`maxTokens`).

- **`AiModelAccessRoute`** (`ai_model_access_routes`): `AiModelId`, `accessRouteName` (an

`AI_ACCESS_ROUTE.*.NAME`), `targetModelName` (the vendor id on that route), `isDefault`, `isActive`.

- **`AiModelToolAssignment`** (`ai_model_tool_assignments`): which tools a model may use (see the

`hor-ai-prompt-document-store` skill for `AiTool`).

- **API keys / base URLs** come from `@openreachtech/renchan-env` via `app/globals/env.js` (e.g.

`CLAUDE_API_TOKEN`, `CLAUDE_API_BASE_URL`, and the analogous OpenAI/Gemini vars) — never hard-code a key.

Only the route that uses a credential reads it, so each route's env vars are listed in

[access-routes.md](./references/access-routes.md).

- **Vertex AI uses a service account, not a key**: Application Default Credentials supply it, and only the

project id and location come from env. Keep a service-account key file out of the repository and the DB.

- **Bedrock uses AWS credentials**: the AWS default credential chain supplies them (prefer a role attached to

the runtime), and only the region comes from env. Never set `AWS_BEARER_TOKEN_BEDROCK` where IAM

credentials should sign.

`AI_ACCESS_ROUTE` names the routes: `DIRECT { NAME: 'direct' }`, `VERTEX_AI { NAME: 'vertex-ai' }` and

`BEDROCK { NAME: 'bedrock' }`.

Model/provider/route constants live in `constants/aiConstants.cjs` (see the **`hor-constant-definition` skill** for the

two-file `.cjs` master + ESM bridge rule).

## Cross-cutting rules

- **Callers never branch on the provider** — resolve a processor by name and use the uniform

`#sendRequestToAi()` / `AiModelResponse` surface.

- **A route owns the client and the credentials, and a processor never reads a key.** Reading a

credential outside the route that uses it spreads one secret across several classes.

- **A concrete model processor adds no logic beyond `#get:aiModel`** — put shared behavior on the provider

base, per-vendor behavior on the provider base too, and model specifics in the DB.

- **Return `null` / a normalized error response, not `undefined`**, on failure; check `#hasError()` before

reading content.

- One class per file, one `export default`, `static create()` factory; getters return a constant.

## Detail files

- [access-routes.md](./references/access-routes.md), the route contract in full, `BulkAiAccessRoutesLoader`,

the `AiModelAccessRoute` rows, and the env vars and authentication each route needs.

- [provider-internals.md](./references/provider-internals.md) — the three provider bases' function-call

feedback shapes and file-upload models (Claude/OpenAI/Gemini), the payload generators

(tools/toolChoice/history trimming/file source), the `AiModelResponse` + capsule method surface, and the

low-level Launcher/Payload/Capsule/Client pattern with where API keys are read.