llms-full.txt · git:20260915.16cf4a2 · 2026-09-15 · sha256 28930b544ff3b9ad
llms-full.txt git:20260915.16cf4a2B
Immutable. This exact content is served forever at /api/v1/blob/28930b544ff3b9ad.
<project title="Apra Fleet" summary="AI-managed fleet orchestration for Claude Code, Gemini, and Antigravity (agy) -- run, update, and coordinate multiple LLM agents from a single hub.">
<docs>
<doc title="Readme" desc="Project overview, installation, member registration, and day-to-day usage for operators.">
<div align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="assets/marketing/hero-banner-dark.svg">
<img src="assets/marketing/hero-banner-light.svg" alt="apra-fleet: run a fleet of AI agents across your devices, your providers, your workflows." width="100%">
</picture>
# apra-fleet
**Run a fleet of AI agents across your devices, your providers, your workflows.**
What Kubernetes did for containers, apra-fleet does for AI agents:
scheduling, credentials, isolation, and observability for an agentic
workforce -- on any machine, anywhere, using every LLM provider at once.
[](https://github.com/Apra-Labs/apra-fleet/actions/workflows/ci.yml)
[](https://opensource.org/licenses/Apache-2.0)
[](https://github.com/Apra-Labs/apra-fleet/releases)
[](https://modelcontextprotocol.io)
[](https://deepwiki.com/Apra-Labs/apra-fleet)
[Quick Start](#quick-start-5-minutes) - [Live Demo](#watch-a-fleet-work) - [How It Works](#how-it-works) - [fleet-sprint Getting Started Guide](docs/fleet-sprint-getting-started.md) - [Website](https://apra-labs.github.io/apra-fleet)
</div>
---
<img src="assets/marketing/dashboard-demo.gif" alt="apra-fleet fleet-sprint dashboard, real recording: the sprint's own integration tester finds two real bugs, files them against itself, then a second cycle plans, fixes, and closes them -- captions burned in." width="100%">
> **This repository is built by the product you are looking at.** An
> autonomous apra-fleet workflow plans, codes, reviews, tests, and ships
> this codebase in multi-hour sprints -- filing bugs against itself and
> fixing them. The recording above is a real run, not a mockup.
---
## Why a fleet?
Running one AI agent is a demo. Running fifty -- across a MacBook in the
office, a GPU box in the lab, three cloud VMs, and your CI -- is an
operations problem nobody else has solved:
- **Which machine runs which agent?** Real devices, not throwaway sandboxes:
registered, credentialed, health-checked members you already own.
- **Which model does which job?** Claude for review, a cheap tier for
mechanical edits, a local vLLM model for private data -- all in one fleet,
routed by cost tier, switchable per task.
- **Who watches the agents?** Durable workflows with supervisors, watchdogs,
reservations, and live dashboards. Agents that die get detected. Work that
stalls gets resumed. Nothing runs silently.
- **Who holds the keys?** Secrets entered out-of-band, never visible to any
model. Per-provider permission composition. Network egress policy per
credential.
One control plane. Any device. Any model. Any workflow. Any domain.
## What you get
| Pillar | Concretely |
|---|---|
| **Any device** | Register any Windows / macOS / Linux machine (local or over SSH) as a fleet member in one command. Cloud members auto-start on demand. Windows members are fully supported for dispatch, command execution, and background long-running tasks (launched detached via WMI). |
| **Any model** | Claude, Codex, Copilot, Antigravity, local models (any OpenAI-compatible endpoint via OpenCode) -- mixed freely. Tier-based routing (cheap / standard / premium) keeps cost governance built in. Cross-provider review is a quality mechanism: a different model, with different blind spots, checks every change. |
| **Any workflow** | Workflows are durable programs, not prompt chains: multi-hour, resumable, observable, with member reservations and atomic state. Write your own; ship it to the fleet. |
| **Any domain** | Not just software development. The pattern fits wherever work decomposes into agent-sized pieces that need orchestration and an audit trail: nightly retail replenishment (reconcile inventory deltas, draft purchase orders for sign-off), logistics exception handling (triage a delayed shipment, re-book, notify), healthcare intake (summarize referrals, check completeness, route), back-office runs (invoice matching, compliance evidence collection). Software engineering is the vertical running today -- your domain is a workflow away. |
<picture>
<source media="(prefers-color-scheme: dark)" srcset="assets/marketing/fleet-topology-dark.svg">
<img src="assets/marketing/fleet-topology-light.svg" alt="apra-fleet topology: one control plane dispatching to six heterogeneous member devices across providers and operating systems" width="100%">
</picture>
## Watch a fleet work
Our flagship workflow, **fleet-sprint**, develops software autonomously:
plan -> develop -> review -> deploy -> integration-test -> harvest, in
cycles, until the goal is met or the evidence says stop.
It is not a toy. It builds apra-fleet itself:
- Multi-cycle sprints running for hours, unattended
- 2,300+ unit tests and an 81-file integration suite against real backends
- Files bugs against itself, decomposes them, fixes them, and blocks its
own release until quality gates pass
- Every dispatch, verdict, and dollar visible live on the dashboard
- Every sprint's raw child stdout/stderr is captured to a per-sprint log
and linked from the dashboard, so a run's output is traceable even if it
crashes before reporting anything back
A fleet that has run in production:
```
pm-1 Opus (premium) orchestrator
doer-1 Sonnet (standard) feature work
doer-2 Antigravity large-context tasks
reviewer Opus (premium) final review
```
The engine does not know what a "sprint" is; it knows how to run your
workflow reliably across your fleet (see **Any domain** above).
## Quick start (5 minutes)
**1. Install** -- one command via npm (Node.js 22+), or grab the
standalone installer binary for your platform from
[Releases](https://github.com/Apra-Labs/apra-fleet/releases) and
double-click it (installation is the default action):
```bash
npm install -g @apralabs/apra-fleet
apra-fleet install # installs for Claude Code (default)
apra-fleet install --llm agy # or --llm opencode / codex / copilot
cd ~/.apra-fleet/bin && apra-fleet start # start the apra-fleet
```
**2. Connect your agent.** Load the fleet server in Claude Code with
`/mcp` (or restart your provider CLI). Your agent now has a fleet.
**3. Register members -- in plain language.** apra-fleet is driven
conversationally through any MCP-capable agent:
> "Register a local member called `doer`. Register another called
> `reviewer`. Pair them."
> "Register 192.168.1.10 as `build-server`. Username akhil, work folder
> `/home/akhil/projects/myapp`."
Remote passwords are collected out-of-band -- typed into a separate
terminal, never the chat -- used once to set up SSH keys, then forgotten.
**4. Run your first workflow:**
```bash
apra-fleet workflow hello-world
```
Then point the fleet at real work:
```bash
apra-fleet workflow fleet-sprint \
--issue my-project-epic --members doer \
--branch fleet-sprint/first-run --base main
```
Open the dashboard, watch your fleet PLAN->BUILD->REVIEW->TEST->SHIP in a loop till closure
> **New to fleet-sprint?** Read the
> [fleet-sprint Getting Started Guide](https://apra-labs.github.io/apra-fleet/fleet-sprint-getting-started.html)
> ([Markdown](docs/fleet-sprint-getting-started.md) if you're reading this on
> GitHub -- [PDF](docs/fleet-sprint-getting-started.pdf)) -- a plain-English walkthrough
> of what it does, what you need to prepare (beads backlog, `deploy.md`,
> test playbooks, member registration), how to launch and monitor a sprint,
> and what's automated versus what's still your call.
**Running fleet-sprint after npm install:** the `apra-fleet workflow
fleet-sprint ...` command above is the same one command for everyone --
whether you installed via `npm install -g @apralabs/apra-fleet`, the
standalone binary, or a git-clone dev checkout. There is no separate
`fleet-sprint` command to install or remember. See
[the full flag reference](packages/apra-fleet-se/fleet-sprint/docs/README.md)
for every option.
## How it works
### Layered Architecture
<picture>
<source media="(prefers-color-scheme: dark)" srcset="assets/marketing/fleet-stack-dark.svg">
<img src="assets/marketing/fleet-stack-light.svg" alt="apra-fleet layered architecture stack: dependencies from OS primitives up to autonomous engineering orchestrators" width="100%">
</picture>
**Component Docs:** [fleet-sprint](packages/apra-fleet-se/fleet-sprint/docs/README.md) | [auto-sprint.js](packages/apra-fleet-se/apra-pm/docs/sprint-workflow.md) | [apra-fleet-client](packages/apra-fleet-client/docs/overview.md) | [apra-pm](packages/apra-fleet-se/apra-pm/README.md) | [apra-fleet-mcp](docs/mcp-tools.md) | [Agent Roles](packages/apra-fleet-se/docs/role-contracts.md)
### Fleet Dispatch Topology
```mermaid
flowchart LR
CP["Control Plane<br/>(Server, Engine, Supervisor)"] -->|Dispatch & Sync| M1["MacBook<br/>(Claude)"] & M2["Linux GPU<br/>(vLLM)"] & M3["Cloud VM<br/>(AGY)"] & M4["Windows<br/>(OpenCode)"]
```
- **Fleet server**: the control plane. Registers members, dispatches commands and prompts, moves files, brokers credentials. Speaks MCP, so any MCP-capable agent can drive a fleet. `execute_prompt` supports session forking (`fork`) on fork-capable providers -- branch a new, independent session from an existing one's context (e.g. a primed session reused across per-task dispatches) without continuing to write into the source session. See [docs/mcp-tools.md](docs/mcp-tools.md#execute_prompt) for the parameter contract.
- **Members**: real machines running provider CLIs. Composes provider-native permissions before every dispatch; unattended modes are scoped, never blanket.
- **Workflow engine**: runs workflow programs with phases, retries, turn budgets, resumable sessions, per-activity persistent state, and a cooperative pause/resume gate any workflow can hook into.
- **Supervisor**: always-on layer -- launch, pause/resume, & stop sprints over HTTP, member reservation ledger, crash watchdog (including a live "paused" state and base-branch-drift indicator), run history.
## Knowledge Layer
Every agent session starts by calling `kb_session_prime`. The KB checks which
files have changed since last read and returns exactly those. Unchanged files
are served from cached summaries -- no re-read, no wasted tokens.
```
Cold session: kb_session_prime returns stale_files=[a.ts, b.ts, c.ts]
Agent reads all three, calls kb_capture for each.
Warm session: kb_session_prime returns stale_files=[], session_warm=true
Agent works from KB summaries. Zero file reads.
```
MCP tools that ship with the KB:
| Tool | What it does |
|------|--------------|
| `kb_session_prime` | Prime a session: stale files, fresh summaries, GitNexus call list |
| `kb_capture` | Store a learning, context-cache, runbook, or knowledge entry |
| `kb_query` | Two-level FTS retrieval (L1: title+summary, L2: full content) |
| `kb_list` | Audit-list entries by confidence/type/module/symbol (read-only, no use_count bump) |
| `kb_context` | Batch file freshness check (single git call for N files) |
| `kb_invalidate` | Mark files stale immediately (also called by the git hook) |
| `kb_promote` | Advance confidence: UNVERIFIED -> INFERRED -> CONFIRMED |
| `kb_harvest` | Extract learnings from a session transcript (auto-fires after execute_prompt) |
| `kb_export` | Write live CONFIRMED entries to `.fleet/kb-canonical.json` -- the git-shareable team bible |
| `kb_setup` | Install git hook, write provider config, store remote token encrypted |
Every KB tool call is scoped to the repo it is about -- a fleet server
handling many members across many repos never lets one repo's learnings land
in another repo's KB. Scope is normally derived from the caller's repo path;
tools also accept an explicit `repo_remote_url` so a remote member (whose
work folder is a path on another host, unreachable from the fleet server's
filesystem) resolves to the same project KB as a local clone of that repo
instead of a shared fallback database. The automatic post-prompt harvest and
the `code_context` KB enrichment path both forward this URL too, and an
unreachable work-folder path is never silently swapped for the fleet
server's own working directory -- see
[Per-repo KB isolation](docs/knowledge-layer.md#per-repo-kb-isolation) for
the full anchor and cache-keying rules.
The backend is swappable: start with local SQLite, add a central HTTP server for
a team, or plug in Postgres later -- all via a one-line config change.
See [docs/knowledge-layer.md](docs/knowledge-layer.md) for the full guide.
## Explore with agents. Operate with programs.
There are two ways to orchestrate agents, and apra-fleet is built on the
observation that you need both -- at different stages of a workflow's life:
- **Exploration mode.** While a workflow is still being discovered, let an
LLM orchestrate: flexible, adaptive, and token-hungry -- every step is a
decision, and every decision costs thinking.
- **Operation mode.** Once you know what must happen, the control flow
becomes a deterministic workflow program. Shell, git, and file steps run
through `execute_command` -- zero tokens. The model is invoked only at
the corners that genuinely require judgment (`execute_prompt`): review
this diff, plan this backlog, decide this exception.
<picture>
<source media="(prefers-color-scheme: dark)" srcset="assets/marketing/cost-collapse-dark.svg">
<img src="assets/marketing/cost-collapse-light.svg" alt="Cost per apra-fleet e2e run: four real LLM-driven runs ranging $0.46-$3.05, then ~$0.00 / run forever after switching to a deterministic workflow." width="100%">
</picture>
That is not a projection -- it is this repository's own e2e setup+teardown
step, before and after we hardened it. Development tokens are not
operating tokens: pay once to discover the workflow, then run it free.
| | LLM-orchestrated (explore) | Workflow-orchestrated (operate) |
|---|---|---|
| Control flow | the model decides each step (tokens) | deterministic program (free) |
| Shell / git / file steps | narrated through the model | `execute_command`, zero tokens |
| Where the model runs | everywhere | judgment nodes only (`execute_prompt`) |
| Cost curve | scales with every step | scales with thinking only |
| Failure mode | drift and silent retries | typed errors, resumable state |
The collapse is two-dimensional. As a workflow hardens, control flow moves
from model to program -- and the judgment nodes that remain move from
frontier models to cheaper ones, because a well-specified task no longer
needs discovery-grade reasoning. **Develop a workflow with Claude;
operationalize it on OpenCode against a local or OpenRouter model.** Same
fleet, same workflow -- swap the members. Tier routing makes it a
registration change, not a rewrite.
Only a fleet makes that trade possible. Single-provider tools cannot leave
their vendor; in-process frameworks cannot move orchestration out of the
token path. Because apra-fleet's unit of execution is the member -- a
machine plus a provider, swappable at registration -- the same hardened
workflow runs on frontier models the day you design it and on commodity
models every day after.
fleet-sprint is this principle, lived: it began as LLM-orchestrated
exploration; each discovered pattern was hardened into the deterministic
engine; today the engine drives hour-long autonomous runs in which models
are consulted only as planner, doer, reviewer, tester, and harvester.
## Compare to alternatives
| Tool | Overlap | Where apra-fleet differs |
|------|---------|--------------------------|
| Single-agent coding assistants | AI writes code | A fleet adds agents that review, test, and deploy each other's work -- across vendors. |
| CI self-hosted runners | Runs work on other machines | Conversational and stateful, not pipeline-triggered; agents carry context between phases. |
| SkyPilot / dstack | Multi-machine compute | Coordinates agents and their context, credentials, and permissions -- not just jobs. |
| Google A2A | Agent-to-agent messaging | An opinionated orchestration and operations layer, not just a transport. |
| Agent frameworks (LangGraph, CrewAI, ...) | Multi-agent logic | Those compose agents inside one process; apra-fleet operates agents across real machines, providers, and days-long workflows. |
When NOT to use it: a one-off single-file change needs no fleet.
## Security model, in one paragraph
Secrets are entered out-of-band into a credential store and referenced as
`{{secure.NAME}}` -- resolved server-side at execution, never visible to
any LLM or log. Credentials scope to members, expire on TTL, and can carry
a network egress policy (allow / deny / confirm). Every member runs with
composed, provider-native permission files -- allow-listed tools, not
god-mode. VCS access is provisioned and revocable per member, across GitHub,
Bitbucket, and Azure DevOps -- host differences (URL shape, PR REST dialect,
auth pattern, error vocabulary) are hidden behind a per-provider descriptor
rather than leaking into shared code; see
`docs/design-azure-devops-vcs-auth.md` for the Azure DevOps provider's
credential-assembly and PAT-lifetime details. A credential-requiring VCS
command can be handed to the server for execution (`vcs_credential_exec`)
rather than the orchestrator learning the plaintext token itself: the
server substitutes the credential into the command, runs it on the member,
and redacts the token from every field of the result -- the plaintext never
transits an orchestrator-readable output. Permission
composition verifies its own delivery: a grant is read back off the target
member and structurally compared against what was intended before it is
reported as applied, so a failed or partial write is surfaced as an
explicit failure rather than a false success.
## Email Configuration
The fleet `send_email` tool sends email via **SendGrid** or **SMTP**. Secrets
are stored in the fleet credential store. Non-secret config (provider, host,
port, from address) is passed by the workflow in each call.
### Storing secrets (one-time setup)
Store email secrets via the CLI:
```bash
# SendGrid API key
apra-fleet secret --set sendgrid_api_key --persist
# SMTP password
apra-fleet secret --set smtp_password --persist
```
Or via the MCP tool (the path an LLM agent uses):
```json
{ "name": "sendgrid_api_key", "prompt": "Enter your SendGrid API key", "persist": true }
```
Secrets are encrypted in the fleet credential store. They never appear in
workflow code, config files, or environment variables.
### Sending email from a workflow
The workflow passes non-secret config inline and calls `send_email`. Load
your config however you prefer (JSON file, hardcoded, etc.):
```javascript
import { parseToolJson } from '@apralabs/apra-fleet-client';
import { connectFleet } from '@apralabs/apra-fleet-client/server-resolution';
const { fleetApi } = await connectFleet({ env: process.env });
// fleetApi wrappers return the raw MCP tool result ({ content: [...] });
// parseToolJson extracts the JSON payload.
const result = parseToolJson(await fleetApi.sendEmail({
provider: 'smtp',
host: 'smtp.example.com',
port: 587,
user: 'notifications@example.com',
from: 'noreply@example.com',
to: 'team@example.com',
subject: 'Sprint Report',
body: 'All tasks completed.'
}));
console.log(`Sent: ${result.messageId}`);
```
The SMTP password resolves from the credential store automatically. It never
appears in the workflow. See `examples/workflows/email-notify/` for a
complete runnable example and `docs/email-workflow-guide.md` for the full
walkthrough.
### send_email Tool Reference
| Parameter | Type | Required | Description |
|---|---|---|---|
| `provider` | `"sendgrid"` or `"smtp"` | no (default: `"sendgrid"`) | Email provider |
| `from` | string | yes | Sender email address |
| `host` | string | SMTP only | SMTP server hostname |
| `port` | number | no (default: 587, or 465 when `secure` is true) | SMTP server port |
| `user` | string | SMTP only | SMTP username |
| `secure` | boolean | no (default: false) | Implicit TLS (port 465). When false, STARTTLS is required. |
| `to` | string or string[] | yes | Recipient email address(es) |
| `subject` | string | yes | Email subject line |
| `body` | string | yes | Plain-text email body |
| `html` | string | no | HTML email body |
| `cc` | string[] | no | CC recipient addresses |
| `bcc` | string[] | no | BCC recipient addresses |
| `attachments` | attachment[] | no | File attachments (base64-encoded) |
Each attachment: `filename` (string), `content` (string, base64), `contentType` (string, optional).
Secrets are resolved from the credential store by name:
- **SendGrid:** `sendgrid_api_key`
- **SMTP:** `smtp_password`
Returns: `{ ok: true, messageId }` on success, `{ ok: false, error }` on failure.
## The packages
| Package | What it is |
|---|---|
| `apra-fleet` | The fleet platform: server, CLI, member management, credentials, workflows runtime |
| `packages/apra-fleet-se` | The software-engineering vertical: fleet-sprint engine, agent contracts, integration suites |
| `packages/apra-fleet-workflow` | Workflow authoring runtime: state, viewer, checkpointing |
| `packages/fleet-api-contract` | Typed API contract shared by server and clients |
## Status and roadmap
apra-fleet is under active development -- by its own fleet. Current focus:
hardening autonomous sprint execution (the toughest workflow we know of),
supervisor-orchestrated multi-sprint operation, and the workflow SDK for
third-party verticals.
## Documentation
| Topic | Link |
|-------|------|
| **fleet-sprint Getting Started Guide (start here, plain English)** | [Website](https://apra-labs.github.io/apra-fleet/fleet-sprint-getting-started.html) - [Markdown](docs/fleet-sprint-getting-started.md) - [PDF](docs/fleet-sprint-getting-started.pdf) |
| Codebase wiki (architecture, internals, AI Q&A) | [DeepWiki](https://deepwiki.com/Apra-Labs/apra-fleet) |
| Install, uninstall, the `--llm` flag | [docs/install.md](docs/install.md) |
| Choosing a provider (roles, gotchas, mixing providers, OpenCode/local models) | [docs/provider-guide.md](docs/provider-guide.md) |
| Transport, service mode, and supported interfaces | [docs/transport-and-service-mode.md](docs/transport-and-service-mode.md) |
| Cost model (tiering, shell-over-prompts, measured token spend) | [docs/cost-model.md](docs/cost-model.md) |
| The PM skill (doer-reviewer sprints, `/pm` commands) | [docs/pm-skill-overview.md](docs/pm-skill-overview.md) |
| FAQ | [docs/FAQ.md](docs/FAQ.md) |
| Troubleshooting | [docs/troubleshooting.md](docs/troubleshooting.md) |
| Keeping Fleet updated (`apra-fleet update`) | [docs/features/update.md](docs/features/update.md) |
| Live member activity (`apra-fleet watch`, `logging.previewChars`) | [docs/features/watch.md](docs/features/watch.md) |
| Secure credentials and passwords | [docs/features/oob-auth.md](docs/features/oob-auth.md) |
| Member category and tags | [docs/features/member-tags.md](docs/features/member-tags.md) |
| Enabling SSH on a remote machine (if it does not have it yet) | [docs/ssh-setup.md](docs/ssh-setup.md) |
| Git authentication | [docs/design-git-auth.md](docs/design-git-auth.md) |
| Cloud compute | [docs/cloud-compute.md](docs/cloud-compute.md) |
| Architecture | [docs/architecture.md](docs/architecture.md) |
| Windows shell selection (probe order, gitbash/pwsh7/powershell5, shell vs os) | [docs/windows-shell-selection.md](docs/windows-shell-selection.md) |
| Cross-shell command construction for member-bound commands | [docs/cross-shell-command-construction.md](docs/cross-shell-command-construction.md) |
| Knowledge Layer (setup, usage, provider swap) | [docs/knowledge-layer.md](docs/knowledge-layer.md) |
| Code intelligence provider abstraction | [docs/code-intelligence-providers.md](docs/code-intelligence-providers.md) |
| Hub-spoke cloud migration plan (historical; see tier-3 ownership ADR) | [docs/hub-spoke-master-plan.md](docs/hub-spoke-master-plan.md) |
| Tier-3 ownership decision (fleet-dashboard vs `src/hub-service/`) | [docs/adr-tier3-ownership.md](docs/adr-tier3-ownership.md) |
| Shared hub/dashboard API contract package | [packages/fleet-api-contract/README.md](packages/fleet-api-contract/README.md) |
| Workflow engine internals (`agent()`/`parallel()`/`pipeline()`, journal, budget, pause/resume) | [packages/apra-fleet-workflow/docs/apra-fleet-workflow-architecture.md](packages/apra-fleet-workflow/docs/apra-fleet-workflow-architecture.md) |
| Cooperative workflow pause/resume (engine, viewer, supervisor, fleet-sprint) | [docs/features/workflow-pause-resume.md](docs/features/workflow-pause-resume.md) |
| Supervisor dashboard live-refresh (`/state` + `/events` SSE, tab-activation refresh, in-memory scope expansion) | [docs/features/supervisor-dashboard-live-refresh.md](docs/features/supervisor-dashboard-live-refresh.md) |
| Writing and running workflow scripts | [packages/apra-fleet-workflow/docs/workflow-guide.md](packages/apra-fleet-workflow/docs/workflow-guide.md) |
| Authoring a SEA-embedded `apra-fleet workflow` (manifest, entry contract, launcher env vars) | [docs/authoring-workflows.md](docs/authoring-workflows.md) |
| Workflow launcher fleet-server resolution order (HTTP singleton vs. stdio) | [docs/adr-workflow-server-resolution.md](docs/adr-workflow-server-resolution.md) |
| Running fleet-sprint (full flag reference; identical for npm-install, standalone binary, and git-clone dev checkout) | [packages/apra-fleet-se/fleet-sprint/docs/README.md](packages/apra-fleet-se/fleet-sprint/docs/README.md) |
| Auto-sprint overview (autonomous plan-develop-review-publish loop) | [packages/apra-fleet-se/docs/overview.md](packages/apra-fleet-se/docs/overview.md) |
| Auto-sprint CLI reference | [packages/apra-fleet-se/docs/cli-reference.md](packages/apra-fleet-se/docs/cli-reference.md) |
| Auto-sprint internals (cycle loop, stall detection, budget, topology) | [packages/apra-fleet-se/docs/architecture.md](packages/apra-fleet-se/docs/architecture.md) |
| Auto-sprint agent role contracts | [packages/apra-fleet-se/docs/role-contracts.md](packages/apra-fleet-se/docs/role-contracts.md) |
| fleet-supervisor skill (start/stop/restart/auto-start-on-boot, sprint launch via HTTP API) | [packages/apra-fleet-se/fleet-sprint/skills/fleet-supervisor/SKILL.md](packages/apra-fleet-se/fleet-sprint/skills/fleet-supervisor/SKILL.md) |
| MCP client SDK overview (transports, `ApraFleet` API) | [packages/apra-fleet-client/docs/overview.md](packages/apra-fleet-client/docs/overview.md) |
| MCP client SDK API reference | [packages/apra-fleet-client/docs/api-reference.md](packages/apra-fleet-client/docs/api-reference.md) |
| MCP client SDK getting started | [packages/apra-fleet-client/docs/getting-started.md](packages/apra-fleet-client/docs/getting-started.md) |
## Community
- Questions and ideas: [GitHub Discussions](https://github.com/Apra-Labs/apra-fleet/discussions)
- Releases: [GitHub Releases](https://github.com/Apra-Labs/apra-fleet/releases)
- Issues: [GitHub Issues](https://github.com/Apra-Labs/apra-fleet/issues)
- What is planned next: [ROADMAP.md](ROADMAP.md)
If Apra Fleet helped you ship faster with better quality, please
[star the repo](https://github.com/Apra-Labs/apra-fleet) -- it helps others
find it.
## Development
Build from source (also the path for Intel Macs):
```bash
git clone https://github.com/Apra-Labs/apra-fleet && cd apra-fleet
npm install && npm run build && npm test
```
See [CONTRIBUTING.md](CONTRIBUTING.md) to contribute.
## License
Apache 2.0 -- see [LICENSE](LICENSE).
---
<div align="center">
**Stop babysitting agents. Start operating fleets.**
[Quick Start](#quick-start-5-minutes) - [GitHub Issues](https://github.com/Apra-Labs/apra-fleet/issues) - [Apra Labs](https://apralabs.com)
</div>
</doc>
<doc title="Vocabulary" desc="Shared terminology -- member, task, skill, PM, fleet, doer/reviewer pattern.">
<!-- llm-context: Defines the terminology used throughout apra-fleet -- member, fleet, PM, doer, reviewer, provider, session, etc. Consult this first when you encounter an unfamiliar fleet-specific term to avoid misinterpreting user requests. -->
<!-- keywords: vocabulary, terminology, member, fleet, PM, doer, reviewer, provider, session, agent, orchestrator -->
<!-- see-also: architecture.md (how these concepts relate), ../README.md (practical usage) -->
# Fleet Vocabulary
## The Problem
"Agent" is overloaded:
1. A fleet member (registered machine/folder that does work)
2. A background Claude process the PM spawns to coordinate
"The agent is running" -- which one? This causes real confusion in logs, conversation, and status updates.
## Approach: Names Over Nouns
Most of the time, use the **specific name** and drop the category word entirely:
- "Sent to dev2" -- not "sent to member dev2"
- "review1 passed PR #13" -- not "reviewer member passed"
- "dev1 is on main" -- not "the dev1 member is on main"
Names are unambiguous. Category nouns are noise when the name is present.
## When You Need the Category
For generic references ("list all ___", "register a new ___"), use:
| Term | Meaning |
|------|---------|
| **member** (or **worker**) | A registered fleet member. The thing that does the work. |
| **subagent** | A background Claude process spawned by the PM. Ephemeral. |
| **session** | A conversation thread on a member. Context persists within it. |
| **fleet** | The collection of all registered members. |
| **PM** | The Project Manager -- the master Claude instance that orchestrates everything. |
| **provider** (or **LLM backend**) | The LLM CLI a member uses: `claude`, `codex`, or `copilot`. Each member has exactly one provider, set at registration and changeable via `update_member`. |
## Rules
1. **Prefer the name**: "dev2 rebased" not "the dev2 member rebased"
2. **"Subagent" is always "subagent"** -- never just "agent" when referring to a background Claude process
3. **"Agent" is banned in PM conversation** -- too ambiguous. Use the name or "member/worker" for fleet members, "subagent" for Claude processes.
4. **API keeps `agent_id`** -- backwards compat in code. User-facing language evolves separately.
5. **Provider is a property of a member**, not a conversation topic. Say "dev2 uses Codex" not "dev2 is a Codex agent". The member identity (the name) is what matters; the provider is just how it executes prompts.
## Example Fleet
```
PM (orchestrator)
|
+-- dev1 (apra-focus, local, claude/standard)
+-- dev2 (apra-focus-dev2, remote, codex/standard)
+-- review1 (apra-focus-review, remote, claude/premium)
+-- review2 (apra-focus-review2, remote, copilot)
```
PM spawns **subagents** to interact with **members**. A subagent is ephemeral (dies after task). A member is persistent (registered, has sessions, has state).
Members with different **providers** are interchangeable from the PM's perspective -- same tools, same dispatch pattern, different CLI underneath.
</doc>
<doc title="Architecture" desc="How the fleet hub, MCP server, and members interact at a system level.">
<!-- llm-context: This document explains the internal architecture of apra-fleet -- the MCP server, member registry, SSH transport, session management, and how tools are dispatched. Read this when a user asks how fleet works under the hood, or when debugging connectivity or session issues. -->
<!-- keywords: MCP server, member registry, SSH, transport, session, tool dispatch, child process, local member, remote member, architecture -->
<!-- see-also: ../README.md (getting started), mcp-tools.md (tool details), vocabulary.md (terminology) -->
# Architecture
## Why This Exists
AI coding agents are powerful on a single machine. But real work spans many machines -- a dev server, a staging box, a GPU trainer, a production host. Today, if you want Claude Code working across all of them, you SSH in manually, run prompts one at a time, and copy files by hand. There's no single pane of glass.
Apra Fleet gives one Claude instance the ability to orchestrate many. Register machines, push files, run prompts, monitor health -- all through natural language from your terminal. One master, many members.
## Conceptual Model
The system has three layers of abstraction:
**Fleet** -> **Members** -> **Sessions**
A *fleet* is the collection of all registered machines. A *member* is one machine with a working directory -- the unit you talk to. A *session* is a conversation thread on a member -- Claude remembers context across prompts within a session, and you can reset it to start fresh.
Members come in two flavors:
- **Remote members** communicate over SSH. They can be any machine you can reach -- Linux VMs, macOS servers, Windows boxes.
- **Local members** run on the same machine as the master, in a different folder. No SSH needed. Useful for isolating work into separate project directories without spinning up another machine.
This distinction is hidden behind a **Strategy pattern**: every tool interacts with members through a uniform interface. The strategy implementation (remote via SSH, or local via child process) is selected at runtime based on member type. Tools never know or care which kind of member they're talking to.
## How It Fits Together
```
+----------------------------------------------------+
| Master Machine |
| |
| Claude Code CLI <--stdio--> Apra Fleet Server |
| | |
| +----------+----------+ |
| | Member Strategy | |
| | (uniform interface)| |
| +--+-----------+-----+ |
| | | |
| Remote Strategy Local Strategy |
| (ssh2 + sftp) (child_process + fs) |
| | | |
| SSH | local exec |
+----------------------------------------------------+
| |
+------------+ +--> /other/project/
v (same machine)
+--------------+
| Remote Member |
| (any OS, |
| any provider)|
+--------------+
```
The MCP server speaks **stdio** -- the standard transport for Claude Code MCP servers. Claude sends JSON-RPC tool calls, the server executes them, returns results. No HTTP, no ports to open.
## Layers
The codebase follows a strict layering:
```
index.ts <- MCP server entry point, tool registration
tools/* <- one file per tool, each self-contained
services/* <- core capabilities (strategy, registry, SSH, file transfer)
providers/* <- LLM provider adapters (Claude, Antigravity, Codex, Copilot)
os/* <- OS-specific command builders (Linux, macOS, Windows)
utils/* <- stateless helpers (crypto, shell escaping)
types.ts <- shared data structures
```
Each layer only depends on the layers below it. Tools never import other tools. Services don't know about the MCP protocol.
## HTTP Transport & Interactive Sessions
Alongside the stdio MCP transport, the server exposes a `StreamableHTTPServerTransport` on `POST/GET/DELETE /mcp` (`src/services/http-transport.ts`), bound to `127.0.0.1` only. This is what lets a member's own `apra-fleet` MCP server connect back interactively -- distinct from the subprocess/SSH-driven `execute_prompt` path, which stays subprocess-only. See `docs/hub-spoke-wire-protocol.md` for the full wire-level design.
Each `/mcp` connection carries one of two identities:
- **JWT-authenticated** -- a `Bearer` token (`src/services/jwt.ts`, HS256, signed with `~/.apra-fleet/fleet.key`) verified via the pluggable `TokenIssuer` (`src/services/token-issuer.ts`). The token's `workspace_id` claim is the hard security boundary (never `project_id`, which is an optional non-security label) -- Phase 1 uses a local dev-mode issuer (one machine == one implicit workspace); a hub-era issuer swaps in behind the same interface with no token-shape change.
- **Unauthenticated URL-param fallback** -- a `?member=<id>` query param, trusted only because the server binds to loopback. Legacy friendly-name params are resolved to the member's UUID via the agent registry.
A connected member is tracked in the in-memory `sessionRegistry` (`src/services/session-registry.ts`), keyed on the composite `(workspace_id, member_id)` -- every lookup is workspace-scoped, so a member connected under a different workspace is indistinguishable from "not connected" (existence is never leaked across the boundary). `send_message` (`src/tools/send-message.ts`) uses this registry to push a `notifications/claude/channel` MCP notification to a connected member's live session, flipping its status to `busy`.
That flip is closed by `report_status` (`src/tools/report-status.ts`): a connected member's OWN session calls it (`online` or `idle`) to report it's done responding. There is no `member_id` parameter -- identity comes entirely from the live MCP session the call arrives on, resolved via `sessionRegistry.findBySessionId(extra.sessionId)` (the SDK populates `sessionId` on every tool call's `extra`). This is the tier-2-local status state machine `docs/hub-spoke-wire-protocol.md` section 4 reserves the `presence.member_status` envelope name for, once a hub relays it upward.
Fleet events (`credential:stored`, `task:completed`, `member:status-changed`, `stall:detected`) broadcast only to sessions in the same workspace as the local orchestrator -- never across a workspace wall.
`execute_prompt` (`src/tools/execute-prompt.ts`) itself is dual-path: for a member with NO live interactive session, it behaves exactly as before (subprocess/SSH, unchanged). For a member that IS interactively connected, it routes through the same channel instead of spawning anything -- `send_message` pushes the prompt and the caller awaits the member's `respond_to_message({reply_to, content})` call, correlated purely in-memory by `src/services/pending-responses.ts` (a `msgid` -> pending-promise map, timeout-bound by the same `timeout_s` the subprocess path uses). Mode selection is decided tier-2-locally against this machine's own `sessionRegistry` -- never from caller-side or (future) hub-side state -- so it is unaffected by whether `execute_prompt` is invoked directly or eventually relayed through a hub. This interactive mode is gated to Claude members only: `docs/interactive-injection-provider-survey.md` confirms it is POC-proven on Claude alone (the other five providers are confirmed unsupported or unconfirmed) -- a non-Claude member with a live session (e.g. from `registerMcpEndpoint`, which gives several providers basic MCP tool access) still falls through to the subprocess path.
## Provider Abstraction
Fleet supports five LLM providers -- Claude Code, Google Antigravity CLI (agy), OpenAI Codex CLI, GitHub Copilot CLI, and OpenCode -- plus a sixth null option, `'none'`, for a plain command executor with no LLM at all (`src/providers/none.ts`). Members can mix providers within a single fleet.
### How It Works
Each member has an optional `llmProvider` field (`'claude' | 'agy' | 'codex' | 'copilot' | 'opencode' | 'none'`). When absent, it defaults to `'claude'` for backwards compatibility. Every tool that interacts with the member's LLM CLI resolves the provider via `getProvider(agent.llmProvider)` and delegates CLI-specific concerns to the `ProviderAdapter` interface.
A `'none'` member supports `execute_command` (already fully provider-agnostic, no changes needed) but never `execute_prompt` in either mode -- rejected immediately with a clear error rather than reaching `NoneProvider`'s methods, most of which throw by design (there is no CLI, no prompt, no model to build a command from). `register_member` skips CLI/auth verification entirely for these members, and status/detail views show `compute only` in place of a token count.
```
+----------+ getProvider() +-----------------+
| Tool | --------------------> | ProviderAdapter |
| (generic)| | (per-provider) |
+----------+ +--------+---------+
| supplies:
cliCommand()
buildPromptCommand()
parseResponse()
classifyError()
authEnvVar
processName
...
```
The `OsCommands` layer sits below this: it handles OS-specific shell wrapping (PATH prepend, PowerShell syntax, base64 decode) and delegates CLI-specific parts (binary name, flags, JSON format) to the provider.
### Provider Files
```
src/providers/
provider.ts - ProviderAdapter interface + shared types
claude.ts - ClaudeProvider
agy.ts - AgyProvider
codex.ts - CodexProvider (NDJSON parser)
copilot.ts - CopilotProvider
opencode.ts - OpenCodeProvider (NDJSON parser, local/self-hosted models)
index.ts - getProvider() singleton factory
```
### Optional Capability Methods
Not every provider CLI can do everything -- some capabilities (e.g. session forking) only exist on a subset of providers. Rather than making every `ProviderAdapter` implement a capability it may not be able to honor, or maintaining a separate out-of-band capability registry, the interface declares such capabilities as an **optional method pair**: a `supportsX?(): boolean` check plus the flag/command builder it gates (e.g. `forkFlag?(sourceSessionId): string`). A provider that has not implemented the pair is capability-incapable by default -- there is no third state and no fallback that fakes the capability through some other mechanism. Callers must always check the support method (`provider.supportsX?.() ?? false`) before calling the builder; calling the builder without checking support first is a caller bug, not a case the interface guards against for you. This keeps capability detection colocated with the capability itself (same file, same provider), avoids a growing "which providers support what" table that drifts from the actual adapter code, and lets a capability request against an unsupporting provider fail loudly and specifically (a dedicated terminal error) instead of degrading silently into different, unexpected behavior.
### Mix-and-Match Fleet
A fleet can have members on different providers simultaneously. The PM dispatches work to members by name -- it doesn't need to know which LLM backend each member uses. The fleet server resolves the correct CLI commands per member at runtime.
```
PM (orchestrator, Claude)
|
+-- dev1 (claude, remote)
+-- dev2 (agy, remote)
+-- dev3 (codex, local)
+-- dev4 (opencode, local) <- self-hosted model via Ollama
+-- review (copilot, remote)
```
All five members use the same `execute_prompt` tool call. The tool builds provider-correct CLI commands for each.
### Key Differences Across Providers
- **`max_turns`** - Claude-only. Ignored for Antigravity, Codex, and Copilot.
- **OAuth credential copy** - Claude-only. Non-Claude providers require an API key (`provision_llm_auth` with `api_key`).
- **JSON output format** - Codex emits NDJSON (one event per line). All others emit a single JSON object. Handled transparently by `provider.parseResponse()`.
- **Session resume** - Claude and Antigravity support resuming specific session IDs. Codex and Copilot resume the most recent local session. OpenCode supports session resume via `--session <id>` or `--continue`.
- **OpenCode** - uses any OpenAI-compatible endpoint (Ollama, vLLM). The user provisions the endpoint; Fleet installs the CLI and agents. Model tiers are set per member at registration via `model_tiers` (since models vary by deployment). Agent files are transformed from Claude format to OpenCode format at install time (tools allowlist -> permission map).
- **Antigravity (agy) response capture** - `AgyProvider.parseResponse()` extracts both the reply text and, when the CLI's transcript exposes one, a `conversation_id` that is surfaced as the response's `sessionId` -- mirroring Claude's session-id capture instead of returning an empty session id for a provider that does expose one.
### Terminal-Signal and Dead-Session Detection Invariants
These invariants apply across all providers that can terminate a dispatch without a clean `execute_prompt` return, and are enforced independently of each other as defense-in-depth -- a failure in one layer must not silently convert into a wasted multi-thousand-second wait for the hard dispatch ceiling:
- **Max-turns detection is channel-agnostic.** A CLI can signal "hit the turn limit" through more than one channel in the same transcript stream -- a `type:result` event's `terminal_reason` field, that same event's `subtype` (`error_max_turns`), a distinct standalone `max_turns_reached` transcript event (which can arrive before, or instead of, any result event, e.g. when a hard-timeout kill truncates the stream first), or a plain-text fallback when no result event terminates the stream at all. `src/providers/claude.ts` normalizes every one of these channels to the same `terminalReason: 'max_turns'` on the parsed response, and `src/providers/provider.ts` exposes a single shared classifier (checking the normalized `terminalReason` OR the raw `error_max_turns` subtype) that every call site uses -- there is exactly one place that decides "was this a max-turns termination," never a per-call-site heuristic. This does not add any new MCP reason-enum value; `max_turns_exhausted` already existed and is now reliably reached from every transcript shape that means it.
- **Stall detection is a superset backstop, not a replacement.** The transcript-mtime cross-check described in `docs/stall-detector-resilience.md` (section 7) exists specifically so that a terminal-signal-detection gap (like the max-turns case above, or any future one) cannot regress into the old failure mode of a dead session sitting unkilled until the hard ceiling -- a frozen transcript is killed within the configured inactivity window regardless of whether content parsing understood why it went quiet.
- **A busy-lock rejection is verified against process liveness before being honored.** `execute_prompt`'s in-flight lock (`inFlightAgents`, `src/tools/execute-prompt.ts`) can outlive the process it was guarding (a child reaped without its exit handler firing, or an interactive session's underlying CLI process dying after registration). Before returning a `busy` rejection, the dispatch path probes the locked session's pid for liveness -- locally via a direct signal-0 check, remotely via a fresh independent liveness round trip (never the possibly-wedged channel that produced the stale lock), and via the session registry's last-known pid for interactive sessions -- and self-heals (releases the lock, warns, and proceeds with the new dispatch) only on a **definitive** dead-pid reading. Ambiguity (no pid captured at all, e.g. a dispatch that hasn't reached its pid-capture step yet) is always treated as still-busy, never as staleness evidence, so a self-heal attempt can never race a dispatch that is still starting up. This keeps `fleet_status`'s independently-computed busy/idle view and the dispatch gate's busy/idle view from ever disagreeing for longer than one dead-lock detection cycle.
- **A confirmed stall cancels the in-flight dispatch, not just the remote process.** Killing the remote pid a stalled dispatch was running is necessary but not sufficient -- the MCP `tools/call` that dispatched it can still be sitting on a pending `execCommand()` promise with no way to reject it, so the client falls back to waiting out its own hard deadline even though the server-side work died minutes earlier. The stall path now threads an `AbortController` through `onStall` that is wired into the same strategy-level `execCommand()` abort signal remote strategies already accept, so a confirmed stall settles the pending call immediately with a typed `stalled` dispatch error instead of silently degrading into a multi-thousand-second client-side timeout wait.
- **The client's dispatch deadline must cover every server-side retry the request can trigger, not just the first attempt.** When the server's own inactivity-timeout handling retries a dispatch once with a fresh session (same `timeout_s`/`max_total_s`), the total server-side worst case is a multiple of the value the client's deadline was derived from. A client deadline sized for only the first attempt can fire before the server's own retry-and-report-cleanly path gets a chance to run, which surfaces a raw transport timeout to the caller instead of a clean typed error. The fix is a single shared retry-budget helper that both sides size their deadlines from, so "the client waits at least as long as the server's own worst-case retry sequence" is a property of one function, not something every dispatch call site has to remember to compute correctly.
- **A provider usage/quota limit is a terminal signal in its own right, distinct from `overloaded`.** Every `ProviderAdapter` exposes `detectUsageLimit(result, parsed)` (`src/providers/provider.ts`), which recognizes a plan/quota exhaustion (e.g. Claude's 429) and returns a `UsageLimitSignal { resumeAt, resumeAtSource: 'parsed' | 'guessed', message }` -- never `null` for a genuine limit, since even an unreadable reset time still yields a guessed resume window. `execute_prompt` (`src/tools/execute-prompt.ts`) runs this check immediately after every `provider.parseResponse()` call in the dispatch path (initial dispatch, stale-session retry, server-overloaded retry, orphan recovery, and the workspace-trust self-heal retry), regardless of exit code -- a 0-exit result event whose text carries the limit message is a limit, not a success. A detected signal short-circuits the stale-session and server-overloaded retries below it: a fresh session cannot cure a plan limit, so retrying would only burn the shared retry budget waiting out a window that is already known. The dispatch returns a structured `reason: 'usage_limit'` result carrying the signal verbatim (`structuredContent.usageLimit`) plus the session id when present, so a caller can resume the *same* session once `resumeAt` passes instead of losing context to a fresh one. `packages/apra-fleet-workflow` passes `usageLimit` and `sessionId` through onto `AgentDispatchError.details` unchanged (no policy decisions in that layer), and the fleet-sprint engine's pause/resume policy (`packages/apra-fleet-se/fleet-sprint/usage-limit-controller.mjs`) is the sole consumer that decides how long to pause and when to re-probe.
See `docs/provider-matrix.md` for the full comparison table.
### Multi-member topology (fleet-sprint)
The `apra-fleet-se` fleet-sprint runner (`packages/apra-fleet-se/fleet-sprint/runner.js`) dispatches roles across configured members: the orchestrator issues every `bd` command and the git push/PR, doers round-robin across the doer pool, and the reviewer runs from the reviewer pool.
Two topology modes are supported, selected explicitly when a sprint starts:
- **`legacy` mode** -- no cross-member sync layer. Every `bd` command the orchestrator issues runs against the orchestrator member's beads DB, while each doer's own `bd close` runs against **its** member's DB; the sprint git branch is only meaningful if all members operate on the same working state. This coheres in exactly two topologies: **single-member** (one member does everything) or a **verified shared-workspace fleet** (every configured member resolves to the same checkout/beads DB). Independent per-member checkouts are **not supported** in this mode -- non-orchestrator members would silently work against a beads DB and git branch the orchestrator never sees.
- **`synced` mode** -- an orchestrator-bracketed git+Dolt sync layer wraps every dispatch that reads or writes git-tracked or beads state, reconciling each member's state explicitly instead of assuming it is already shared. This is what makes genuinely independent per-member checkouts safe. The sync layer includes a scripted-first escalation ladder for both git and Dolt conflicts (mechanical detection and resolution first, an agent dispatched with an explicit conflict runbook only as a documented last resort), plus supervisor-owned global coordination (a Dolt push mutex and a child-id allocator) so concurrent cross-member Dolt writes never silently corrupt each other's beads clone.
Guards enforcing whichever mode is selected:
1. **Branch-ensure everywhere** (both modes): before the first doer round, the runner git-ensures the sprint branch (`fetch` + `checkout -B`) on **every** member in the union of the orchestrator/doer/reviewer pools -- not just the orchestrator -- and non-destructively re-checks-out the branch on each member at the start of later cycles (it never resets to base once work is committed).
2. **Topology precondition**: `bin/cli.mjs` calls `checkMemberTopology()` before starting a multi-member sprint. In `legacy` mode it compares an identity signal (`git rev-parse HEAD`) across the configured members and **refuses to start with a clear error** if they disagree (or if a member's signal can't be obtained); in `synced` mode HEADs are allowed to differ, but every member must share the same git remote origin and pass a live beads-Dolt-pull probe. Single-member sprints trivially pass either check.
See `packages/apra-fleet-se/docs/architecture.md` for the full internals of both modes, including the escalation ladders and the always-on supervisor service that launches and tracks sprints.
## Workflow Subsystem (SEA-embedded workflow runner)
`apra-fleet workflow <name>` runs a self-contained script (an ESM entry point
under `workflows/<name>/`) against a live fleet connection, from inside the
single-executable-application (SEA) binary -- no separate Node install and no
unpacking to a temp directory required. `fleet-sprint` is itself shipped as one
of these built-in workflows rather than as a bespoke subcommand, so the
launcher, the packaging, and the docs only need to solve this problem once.
**Two always-separate processes.** The workflow launcher (`apra-fleet
workflow`, `fleet-sprint`) and the `apra-fleet` MCP server are never merged
into one process. This is a hard boundary, not an optimization detail --
see `docs/adr-workflow-server-resolution.md` for the ADR and its explicit
scope guard against reopening it.
**Server connection resolution is a single shared helper**, not duplicated
per consumer. `@apralabs/apra-fleet-client/server-resolution`
(`packages/apra-fleet-client/src/client/server-resolution.mjs`) implements
one resolution order used identically by `src/cli/workflow.ts` and
`packages/apra-fleet-se/bin/cli.mjs`:
1. `APRA_FLEET_TRANSPORT` forced override (`http` fails loud with no
singleton rather than silently falling back to a private server; `stdio`
or a set `APRA_FLEET_SERVER_CMD`/`_BIN` goes straight to self-spawn).
2. Probe for a healthy HTTP singleton (`checkRunningInstance()` -- the same
pid + `/health` check the installed service already uses for
startup-dedup) and attach with zero spawned processes. This is the
default, steady-state path.
3. Fall back to stdio self-spawn (the four-tier command resolution
`resolveFleetServerCommand()` already had) only when no healthy
singleton is found.
The rationale for one shared helper over two copies: a launcher that merely
mirrored the old stdio-only `resolveFleetServerCommand()` would always
self-spawn a private server even when a healthy HTTP singleton already
existed, doubling running servers and splitting state. Duplicating the
resolution order in two languages (TS launcher, MJS fleet-sprint) was
rejected because it guarantees drift between the two copies over time; see
the ADR for the full tradeoff writeup.
**Workflow entry contract:** a workflow is a directory under `workflows/`
containing a `workflow.json` (name, entry, description) and an entry file
that is either self-executing ESM or exports `main(args)` / `run(args)` /
a default export. The launcher sets two env vars for every workflow run --
`APRA_FLEET_SERVER_BIN` and `APRA_FLEET_SE_SCHEMAS_DIR` -- and never
clobbers a value the caller already set. See `docs/authoring-workflows.md`
for the full contract, including the raw-node escape hatch and how a
workflow should import the shared engine/client packages.
**Entry-path escape prevention is security-critical, not incidental:** the
launcher resolves a workflow's `entry` against its own directory and
rejects any resolution that escapes it (checked via `path.relative` plus
`..`/absolute-path detection) before ever executing the file. A workflow
manifest is not a trusted-by-construction input.
**Packaging:** the workflow runtime, agent schemas, and built-in workflows
(including fleet-sprint) are embedded as SEA assets by
`scripts/gen-sea-config.mjs`, proven viable by an earlier spike that
dynamic `import()` of on-disk ESM works from inside a SEA main script on
all three target OSes.
**Install/uninstall/update lifecycle:** `install.ts`'s workflow-install step and its extraction
logic (extract-to-temp-then-rename, with Windows EBUSY retry/backoff) were
factored out into `src/cli/workflow-assets.ts` so more than one caller can
share the exact same code path instead of re-implementing it:
- `apra-fleet install` (fresh install) calls `extractWorkflowSubsystemAssets()`
to lay down `~/.apra-fleet/{node_modules,schemas,workflows/<builtin>}`.
- `src/cli/workflow.ts`'s launcher self-heals: if a `workflow <name>`
invocation finds the on-disk payload missing or incomplete
(`hasWorkflowSubsystemAssets()` is false), it re-extracts from the same
embedded SEA assets before running, rather than failing with an opaque
module-not-found error.
- `apra-fleet uninstall --skill workflows` removes the shared runtime/schema
dirs plus only the built-in workflow subdirectories recorded in
`workflows/.installed.json`'s `builtin` array (falling back to the static
`BUILTIN_WORKFLOW_NAMES` list if that manifest is missing, so a
partially-installed tree still cleans up). User-authored workflow
directories are left in place; `workflows/` itself is only removed if
nothing user-authored remains, and the command prints which user
workflows it kept.
- `apra-fleet update` reads back the previously-persisted `--workflows` mode
and threads it into the re-invoked install, so an update refreshes
built-in workflow assets to the new version while preserving any
user-authored workflows already on disk.
**CI coverage:** `build:binary` smoke tests
exercise the packaged SEA binary's `workflow` subcommand and the
fleet-sprint-as-built-in-workflow path end-to-end (not just the source
`.ts`/`.mjs` files), and a regression-guard test suite
(`tests/regression-command-surface.test.ts`) pins the full existing CLI
command surface (`install --help`, `uninstall --dry-run`, `--version`,
stdio handshake) against golden fixtures so future workflow-subsystem work
can't silently change unrelated command output.
The workflow subsystem is installable, self-healing, uninstallable, and
update-safe end-to-end.
**Cooperative pause/resume** is a generic engine primitive available to any
workflow run (`requestPause()`/`requestResume()`/`setPauseGuard()`), separate
from cancellation (`requestStop()`): a pause is deferred until in-flight work
drains to zero and an optional caller-supplied guard confirms the run is at a
state boundary it considers clean, rather than landing wherever the request
happened to arrive. The per-run viewer and the multi-sprint supervisor
dashboard both surface it, and fleet-sprint is the first workflow to attach
real domain behavior to it (releasing/re-acquiring member reservations and
resyncing git/beads state across a pause). See
[docs/features/workflow-pause-resume.md](features/workflow-pause-resume.md)
for the full design and
[packages/apra-fleet-workflow/docs/apra-fleet-workflow-architecture.md](../packages/apra-fleet-workflow/docs/apra-fleet-workflow-architecture.md)
section 4.7 for the engine-level contract.
**The multi-sprint supervisor dashboard's live-refresh** deliberately reuses
the per-run viewer's own incremental-refresh architecture (a lean JSON state
endpoint plus a change-signal stream driving a debounced client poll) instead
of a second, divergent mechanism -- see
[docs/features/supervisor-dashboard-live-refresh.md](features/supervisor-dashboard-live-refresh.md)
for the full design, including the in-memory scope-expansion fix that removed
a per-graph-node subprocess spawn from every dashboard render.
**Dev-mode manifest bundling is a transitive-dependency, not a top-level-only,
contract.** The dev-mode install path bundles `@apralabs/apra-fleet-client`'s
own package tree so the workflow runtime can load it without a real npm
install. Because that client package can itself gain a runtime dependency
(e.g. for a new transport), the manifest builder must walk and bundle the
client's own `dependencies`, not just the client package's own files -- a
manifest that copies only the client's top-level directory silently ships a
client that throws a module-not-found error the moment it needs a dependency
it never had bundled alongside it. Any future dependency added to
`apra-fleet-client` must be re-verified against this bundling path, not just
against the production/packaged-binary install path, since the two paths
build the manifest differently.
## PM Skill
The PM skill and its role agent definitions (planner, plan-reviewer, doer, reviewer, deployer, integ-test-runner, ci-watcher, harvester), plus their shared `agents/schemas/` and `agents/_shared/` assets, live in this monorepo at `packages/apra-fleet-se/apra-pm/`. At build time, `scripts/dist-pm.mjs` copies the skill and agent files into `dist/` (and `scripts/gen-sea-config.mjs` embeds them as SEA assets for the standalone binary), so all three install paths -- dev-mode, npm, and SEA binary -- carry the same set. At install time, agents are written to the provider's agents directory (e.g. `~/.claude/agents/`); `uninstall` removes that directory in the same pass. For OpenCode members, agent frontmatter is transformed from Claude format to OpenCode format during installation. Claude installs additionally get the `auto-sprint-args` skill (the args contract for the `/auto-sprint` workflow), written to `~/.claude/skills/auto-sprint-args`.
Remote fleet members do not share the operator's home directory, so `register_member`, `update_member`, and (starting with the first dispatch after an orchestrator upgrade) `execute_prompt` each independently hash-diff the canonical agent set against the remote box and push anything missing or stale. See [docs/mcp-tools.md](mcp-tools.md) for per-tool behavior details.
The sprint engine (`packages/apra-fleet-se/fleet-sprint/`) and these role prompts are generic: they drive a sprint against any target repo, and apra-fleet building itself with them is only one target. A mechanical guard keeps apra-fleet-specific assumptions out of their LLM-facing text -- see [docs/generic-engine-boundary.md](generic-engine-boundary.md).
## Key Design Decisions
### Strategy Pattern for Member Types
Rather than scattering `if (agent.agentType === 'local')` checks across every tool, the local/remote distinction lives in a single place: the strategy factory. Tools call `getStrategy(agent).execCommand(...)` and get back the same result shape regardless of how it was executed. Adding a third member type (e.g., Docker containers, cloud VMs with API-based access) means writing one new strategy class -- no tool changes.
### Passwords Encrypted at Rest
SSH passwords are encrypted with AES-256-GCM before being written to the registry file. The encryption key is derived from the machine's identity (hostname + OS username), so the registry file is meaningless if copied to another machine. This isn't meant to stop a determined attacker with root access -- it prevents accidental plaintext exposure in backups, screenshots, or config file shares.
### Connection Pooling with Idle Timeout
SSH connections are expensive to establish (TCP + key exchange + auth). The server pools them in memory and reuses connections across tool calls, with a 5-minute idle timeout that auto-closes unused connections. Timers are `unref()`'d so they don't prevent Node from exiting.
The idle timer is reset by normal pool activity (`execCommand`/`getConnection`), but a stall-detector entry that is still `provisional` (created before it has a log file to poll) does not generate that activity itself -- it only benefits from the stall poller's own incidental probes. The pool's cleanup path therefore never tears down an entry purely on the idle timer firing: it re-checks for an active channel immediately before ending the connection, and only actually closes it when genuinely idle. This closes off a class of latent bug where a long-running dispatch with a provisional-shaped stall entry could otherwise have a live channel reaped out from under it. Separately, pooled SSH connections have keepalive configured (interval + a bounded probe count) so a silently dropped network connection is detected and torn down rather than sitting as a phantom pool entry indefinitely.
### Base64 Prompt Encoding
Prompts sent to remote members are base64-encoded before being passed through SSH. This sidesteps the shell escaping nightmare of nested quoting across SSH -> bash -> claude CLI, across different operating systems. The remote member decodes before passing to Claude.
### Session Persistence
Each member stores an optional `sessionId` -- a Claude conversation thread ID. When `resume=true` (the default), subsequent prompts continue the same conversation, so the remote Claude has full context of prior exchanges. Resetting a session is an explicit action, not an accident.
### File-Based Registry
All fleet state lives in `~/.apra-fleet/data/registry.json` -- a single JSON file in the user's home directory. It's deliberately not in the project directory (won't be git-committed accidentally) and not in a database (no server to run, no migrations). For a fleet of dozens of members, JSON is more than sufficient.
### Duplicate Folder Prevention
Two members cannot share the same working directory on the same device. For remote members, "same device" means same SSH host. For local members, "same device" is always the master machine. This is enforced during registration and updates. It prevents two members from stomping on each other's files.
## Tools
The tools break into natural groups in **[mcp-tools.md](mcp-tools.md)**:
**[Lifecycle](mcp-tools.md#1-lifecycle-tools)** -- `register_member`, `list_members`, `update_member`, `remove_member`, `shutdown_server`
Manage the fleet roster and server lifecycle. Registration validates connectivity, detects the OS, and checks that Claude CLI is available. Removal includes best-effort cleanup of auth credentials on the member.
**[Work](mcp-tools.md#2-work-tools)** -- `send_files`, `receive_files`, `execute_prompt`, `execute_command`, `stop_prompt`, `monitor_task`
The core workflow. Push and pull files, run prompts against a member, run shell commands directly, and stop a running prompt.
**[Infrastructure](mcp-tools.md#3-infrastructure-tools)** -- `provision_llm_auth`, `setup_ssh_key`, `update_llm_cli`
One-time setup and maintenance. Provision auth (copy OAuth credentials or deploy API key for any provider), migrate from password to key auth, update the LLM CLI on members.
**[Observability](mcp-tools.md#4-observability-tools)** -- `fleet_status`, `member_detail`
Two-layer monitoring. `fleet_status` gives a quick summary table across all members with fleet-aware busy detection (distinguishes between Claude processes serving this member vs unrelated Claude activity). `member_detail` drills into one member with connectivity, CLI version, session state, and system resource metrics.
## Cross-Platform Support
Members can run Windows, macOS, or Linux. The `os/*` command builders generate the right shell commands for each OS -- different commands for checking processes, reading memory, and setting environment variables -- while `src/utils/platform.ts` handles OS detection and path resolution. The OS is auto-detected during registration (`uname -s` on Unix, `cmd /c ver` on Windows) and stored in the member record so subsequent tool calls don't need to re-detect.
</doc>
<doc title="Install" desc="Installation, uninstallation, and the --llm/--skill flags.">
# Install, uninstall, and update
This page covers installing Apra Fleet, what the installer writes, controlling
which skills are installed, uninstalling, and self-updating.
## Requirements
- An AI coding agent CLI on the machine where you run Fleet - Claude Code,
Antigravity (agy), Codex, Copilot, or OpenCode.
- SSH access to any remote machines you want to register as members. The local
machine needs nothing extra; remote members need only an SSH server.
## Quick install
Installation is the default action -- just run the binary with no arguments (or double-click it
on Windows).
**macOS (Apple Silicon)**
```bash
curl -fsSL https://github.com/Apra-Labs/apra-fleet/releases/latest/download/apra-fleet-installer-darwin-arm64 -o apra-fleet-installer && chmod +x apra-fleet-installer && ./apra-fleet-installer
```
**Linux (x64)**
```bash
curl -fsSL https://github.com/Apra-Labs/apra-fleet/releases/latest/download/apra-fleet-installer-linux-x64 -o apra-fleet-installer && chmod +x apra-fleet-installer && ./apra-fleet-installer
```
**Windows (x64)** -- download `apra-fleet-installer-win-x64.exe` and double-click it, or run in PowerShell:
```powershell
Invoke-WebRequest -Uri https://github.com/Apra-Labs/apra-fleet/releases/latest/download/apra-fleet-installer-win-x64.exe -OutFile apra-fleet-installer.exe; .\apra-fleet-installer.exe
```
Intel Macs: there is no prebuilt `darwin-x64` binary -- build from source (see
the Development section of the [README](../README.md)).
## Manual install
Download the installer for your platform from
[GitHub Releases](https://github.com/Apra-Labs/apra-fleet/releases):
- `apra-fleet-installer-linux-x64` -- Linux (x86_64)
- `apra-fleet-installer-darwin-arm64` -- macOS (Apple Silicon)
- `apra-fleet-installer-win-x64.exe` -- Windows
Double-click the downloaded file, or run it from the terminal. Installation is the default action:
```bash
# macOS (Apple Silicon) -- no subcommand needed; installation is the default
chmod +x apra-fleet-installer-darwin-arm64 && ./apra-fleet-installer-darwin-arm64
# Linux (x64)
chmod +x apra-fleet-installer-linux-x64 && ./apra-fleet-installer-linux-x64
```
```powershell
# Windows
.\apra-fleet-installer-win-x64.exe
```
> The `install` subcommand is also accepted and does the same thing:
> `./apra-fleet-installer install`.
## What `install` writes
| Path | What it is |
|------|-----------|
| `~/.apra-fleet/bin/apra-fleet[.exe]` | The fleet binary |
| `~/.apra-fleet/hooks/` | Shell hooks (statusline, etc.) |
| `~/.apra-fleet/scripts/` | Helper scripts |
| `~/.apra-fleet/node_modules/` | Shared on-disk workflow runtime (`@apralabs/apra-fleet-workflow`, `@apralabs/apra-fleet-client`, vendored `ajv` + deps) that `apra-fleet workflow <name>` and any user-authored workflow resolve bare specifiers against -- see `docs/authoring-workflows.md` |
| `~/.apra-fleet/schemas/` | Installed agent role verdict/input JSON schemas; the `APRA_FLEET_SE_SCHEMAS_DIR` default the workflow launcher sets |
| `~/.apra-fleet/workflows/` | Installed workflows (`.installed.json` + one directory per workflow, built-in or user-authored); run with `apra-fleet workflow <name> [args...]` -- see `docs/authoring-workflows.md` |
| `~/.claude/skills/fleet/` | Fleet skill (MCP tool docs for Claude) |
| `~/.claude/skills/pm/` | PM orchestration skill |
| `~/.claude/skills/pm/cost.js` | Auto-generated CJS module with sprint cost functions (all providers with PM) |
| `~/.claude/workflows/auto-sprint.js` | Full auto-sprint workflow (Claude only) |
| `~/.claude/skills/auto-sprint-args/` | Args contract for the `/auto-sprint` workflow (Claude only) |
| `~/.claude/skills/fleet-sprint-cli/` | How to launch `apra-fleet workflow fleet-sprint` -- flag contract, preconditions, detached launch (all providers) |
| `~/.claude/agents/` | PM role-agent files (planner, doer, reviewer, etc.), plus `schemas/` and `_shared/` -- written whenever PM is installed and the provider has an agents directory (not codex/copilot) |
For other providers, these are written to that provider's skill/config directories. For example, for Antigravity (`agy`), settings are written to `~/.gemini/antigravity-cli/settings.json`, and hooks / MCP configs are merged into `~/.gemini/config/hooks.json` and `~/.gemini/config/mcp_config.json`.
This local install only covers the machine you run it on. Remote fleet members get their own copy of the PM agent files independently -- `register_member` and `update_member` push them on first contact, and `execute_prompt` re-checks and re-provisions any missing or stale files on first dispatch to that member each server run (so an existing member picks up new agent files after you upgrade Fleet, without needing to be re-registered). Local members are unaffected -- they share the operator's home directory above.
The install also registers the MCP server (`claude mcp add apra-fleet`) and
configures a status bar icon showing fleet member activity.
### Two sprint entry points -- do not confuse them
`apra-fleet` ships **two separate, independently maintained** sprint
implementations:
| | `auto-sprint` (Claude Code workflow) | `fleet-sprint` (apra-fleet CLI workflow) |
|---|---|---|
| Written by | `install` (table above) | `install` populates `~/.apra-fleet/workflows/`; the engine ships as source inside the `@apralabs/apra-fleet` package |
| Providers | Claude Code only | Any provider a fleet member is registered with (Claude, Codex, Copilot, Antigravity/agy, OpenCode) |
| Source package | `packages/apra-fleet-se/apra-pm/.claude/workflows/auto-sprint.js` | `packages/apra-fleet-se` (shipped unbundled as source) |
| Model selection | Literal Claude model names | Fleet's `cheap`/`standard`/`premium` tier keywords, per-member |
| How you run it | `/auto-sprint <bead-ids>` inside a Claude Code session (the Workflow tool) | `apra-fleet workflow fleet-sprint --issue ... --members ... --branch ... --base ...`. See `packages/apra-fleet-se/docs/cli-reference.md` |
There is no separate `fleet-sprint` bin: the root package's `bin` field
contains only `apra-fleet`, and the engine is reached through
`apra-fleet workflow fleet-sprint`. See `docs/npm-packaging.md` for the
shipped package layout and `packages/apra-fleet-se/docs/cli-reference.md` for
the engine's server- and schema-resolution order.
### The `apra-fleet workflow <name>` subcommand
`install` also populates `~/.apra-fleet/node_modules/`, `~/.apra-fleet/schemas/`,
and `~/.apra-fleet/workflows/` (see the directory table above) so that
`apra-fleet workflow <name> [args...]` -- the SEA-binary workflow runner --
can run built-in workflows (`fleet-sprint`, `hello-world`) or any
user-authored workflow with zero system Node required. See
`docs/authoring-workflows.md` for the full authoring contract.
The workflow launcher and the `apra-fleet` MCP server it talks to are
always separate processes. Set `APRA_FLEET_TRANSPORT=http` (the default) or
`APRA_FLEET_TRANSPORT=stdio` to control how the launcher reaches that
server: `http` (default) attaches to the already-running installed-service
singleton at `http://localhost:${APRA_FLEET_PORT:-7523}/mcp` and spawns
nothing; `stdio` self-spawns a private server as a subprocess. See `docs/adr-workflow-server-resolution.md` for the full
resolution order (this same order also governs where role schemas resolve
from in the installed-binary case: `APRA_FLEET_SE_SCHEMAS_DIR`, set by the
launcher to `~/.apra-fleet/schemas`, is tier 1 of the schema resolution
described in `packages/apra-fleet-se/docs/cli-reference.md`).
**What `install` does NOT do:**
- No system-level changes -- no `/usr/local`, no PATH modification, no
admin/sudo required.
- No network calls beyond `claude mcp add` -- the binary stays local.
- No background services or daemons -- the fleet server starts on demand when
your AI coding agent connects.
## The `--skill` flag
By default, `install` writes both the fleet and PM skills. Use `--skill` to
control exactly which skills are installed:
| Flag | Skills installed |
|------|------------------|
| `install` (no flag) | fleet + pm (default) |
| `install --skill all` | fleet + pm |
| `install --skill fleet` | fleet only |
| `install --skill pm` | fleet + pm (pm depends on fleet) |
| `install --skill none` | neither |
| `install --no-skill` | neither (same as `--skill none`) |
## Install for other providers (Antigravity, Codex, Copilot, OpenCode)
By default, `install` configures Apra Fleet for **Claude Code**. Use the `--llm`
flag to install for a different provider instead:
```bash
apra-fleet --llm agy # Google Antigravity CLI
apra-fleet --llm codex # OpenAI Codex CLI
apra-fleet --llm copilot # GitHub Copilot CLI
apra-fleet --llm opencode # OpenCode CLI
apra-fleet --llm claude # Claude Code (the default)
```
The `install` subcommand is also accepted and does the same thing:
`apra-fleet install --llm agy`.
`--llm` decides which provider's configuration the installer writes to. The MCP
server registration, hooks, statusline, permissions, and skills all go into that
provider's config directory -- for example `~/.gemini/antigravity-cli/` for
Antigravity -- instead of `~/.claude/`. To support more than one provider on the
same machine, run `install` once per provider.
`--llm` combines with `--skill`, e.g. `apra-fleet install --llm agy --skill
pm`. Supported values: `claude` (default), `agy`, `codex`, `copilot`,
`opencode`.
After a non-Claude install, load the server by restarting that provider's CLI --
only Claude Code uses `/mcp`.
### Agy note
`apra-fleet install --llm agy` configures Fleet for the Google Antigravity CLI.
Agy uses Google OAuth by default -- a browser-based login flow is required per
machine, so `provision_llm_auth` does **not** work for remote agy members today.
For headless or remote members, set `ANTIGRAVITY_API_KEY` (obtain from
[Google AI Studio](https://aistudio.google.com)) in the environment before
invoking fleet commands. The agy CLI checks env vars before falling back to
OAuth.
## Uninstall
The built-in uninstall command surgically removes MCP registration,
permissions, hooks, status line, skill directories, and PM agent files
(`~/.claude/agents/`, or the equivalent provider directory) without touching
your other settings:
```bash
apra-fleet uninstall
```
| Flag | Effect |
|------|--------|
| `--dry-run` | Preview what would be removed, without modifying anything |
| `--force` | Automatically stop the running fleet server before uninstalling |
| `--yes` | Skip the confirmation prompt |
| `--llm <provider>` | Remove only a specific provider (`claude`, `agy`, `codex`, `copilot`, `opencode`) |
| `--skill fleet\|pm\|workflows\|all` | Remove only the specified skill directories (default: `all`) |
`--skill workflows` removes the shared workflow runtime and schemas
(`~/.apra-fleet/node_modules/`, `~/.apra-fleet/schemas/`) plus only the
built-in workflow subdirectories under `~/.apra-fleet/workflows/` (read from
`workflows/.installed.json`'s `builtin` list, falling back to the static
built-in name list if that manifest is missing). Any user-authored
`workflows/<name>/` directories are left in place, and the command reports
which ones it kept; the `workflows/` root itself is only removed if nothing
user-authored remains in it.
Examples:
```bash
# Preview the full uninstall
apra-fleet uninstall --dry-run
# Full uninstall, stop server automatically
apra-fleet uninstall --force --yes
# Remove only PM skills across all providers
apra-fleet uninstall --skill pm
# Remove only Claude's fleet skills
apra-fleet uninstall --llm claude --skill fleet
# Remove only the workflow runtime + built-in workflows, keep user-authored ones
apra-fleet uninstall --skill workflows
```
If the fleet server is running, uninstall aborts and tells you to re-run with
`--force`. Full detail: [docs/features/uninstall.md](features/uninstall.md).
## Customizing model tier mapping
By default, each provider maps the three tiers (`cheap`, `standard`, `premium`)
to hardcoded model names. You can override any of these per-provider by creating
a `config.json` file in the Fleet data directory:
```
~/.apra-fleet/data/config.json
```
If you set `APRA_FLEET_DATA_DIR`, the file lives at
`$APRA_FLEET_DATA_DIR/config.json` instead.
**Schema example:**
```json
{
"providers": {
"agy": {
"modelMapping": {
"cheap": "GPT-OSS 120B (Medium)",
"standard": "Gemini 3.1 Pro (High)",
"premium": "Claude Opus 4.6 (Thinking)"
}
},
"claude": {
"modelMapping": {
"cheap": "claude-haiku-4-5",
"premium": "claude-opus-4-7"
}
}
}
}
```
Provider keys: `claude`, `codex`, `copilot`, `agy`, `opencode`. Tier keys:
`cheap`, `standard`, `premium`. All fields are optional -- omitted tiers fall
back to the provider's built-in default.
**Precedence:** per-member override (`update_member --model-cheap/standard/premium`)
> user config > hardcoded provider default.
If the file is missing, Fleet proceeds with built-in defaults. If the JSON is
malformed, Fleet logs a warning to stderr and ignores the file.
## Self-update
Update the fleet binary to the latest release:
```bash
apra-fleet update
```
This checks the latest GitHub release, downloads the installer for your
platform, and re-runs it automatically. The server restarts with the new
binary. If you are already on the latest version it reports so and exits. Full
detail: [docs/features/update.md](features/update.md).
### Stopping a running server before an overwrite install
`install --force` (and `update`, which drives the same path) must stop the
currently-running server before copying the new binary over the installed
path, since the OS refuses to overwrite a binary that is still mapped into a
running process. A single termination signal followed by a fixed delay is not
reliable: a singleton that is mid-request can take longer to exit than an
arbitrary fixed sleep, and a copy attempted before it actually exits fails
outright and leaves the old server running.
The install path instead polls process liveness over a bounded grace window
after the initial termination signal, and escalates to a harder kill signal
if the process is still alive once that window elapses, polling again over a
second (shorter) window before giving up. The binary copy is only attempted
once the old process is confirmed gone. Symmetrically, the "stopped running
server" success message is gated on that same confirmation rather than
printed unconditionally -- if the process is still detected running after
both the initial signal and the escalation, install reports a clear error
(with the manual kill command for the platform) and exits non-zero instead of
proceeding into a copy that would fail anyway or claiming success it can't
back up.
**A launchd/systemd/Windows-service-managed server defeats a pkill-first
approach entirely**, not just slows it down. On macOS, the LaunchAgent
installed for the server (`~/Library/LaunchAgents/com.apra-fleet.server.plist`)
is registered with `KeepAlive` set for a non-successful exit, so the service
manager relaunches the server (as a new PID) the instant a `SIGTERM`/
`SIGKILL` reaches it -- signalling the process by name after that point
cannot win the race, since the pid it is tracking is already stale, and a
liveness poll racing the relaunch would report the same "could not stop the
running server" failure no matter how long the grace windows are.
`install --force` avoids this by stopping the registered **service** first,
never signalling the bare process as the first move: it snapshots the
currently-running apra-fleet pids *before* touching anything, then (when a
service is registered) calls the platform `ServiceManager.stop()` for a
graceful shutdown, and only escalates to a direct kill signal if a poll
afterward still finds a process alive **with the same pid it snapshotted
before stopping** -- i.e. nothing relaunched and there is no supervisor race
to lose. If a pid appears that was not in the original snapshot, that is
conclusive evidence of a supervisor relaunch (not a process refusing to
die), and install reports it as exactly that, with the platform's service-
stop command, instead of retrying a signal against a name that will keep
being relaunched forever. The server's own log corroborates this case with
consecutive startup lines under a different PID each time a kill was
attempted. Killing the process without first stopping its service
registration is still not a valid workaround for any code path that has to
solve this problem elsewhere -- it reproduces the exact race above.
If the guard stopped a registered service to win the copy, install restarts
that service again once the new binary is in place -- the stop above exists
only to release the file lock for the overwrite, not to leave the operator's
previously-running server down. This restart is conditional on the service
having actually been stopped by this guard; a plain `apra-fleet install`
run that never touched a running service does not attempt to start one that
was never asked to stop.
### Replaying the npm-publish smoke step locally with an unrelated server running
CI's "Pack + install into a clean temp prefix (fleet-sprint smoke test)" step
packs the CLI and installs it into an isolated, throwaway prefix. That step
(and any local replay of it) is safe to run even while an unrelated
apra-fleet server is already up on the same machine, because the
running-process guard is scoped to the install being performed rather than
to any apra-fleet process anywhere on the OS: it only fires when the running
server's data dir matches the data dir this install targets, or when the
running executable it detects lives under the install prefix being written
(the ETXTBSY case). A server running against a different data dir and a
different install prefix does not trip it.
To replay the step locally, isolate the install the same way CI does by
pointing these env vars at throwaway locations before running
`apra-fleet install`:
- `HOME` (or `USERPROFILE` on Windows) -- so the default data dir and install
prefix resolve under a temp directory instead of your real home.
- `APRA_FLEET_DATA_DIR` -- overrides the data dir directly if you want it
separate from `HOME`.
- The install prefix (where the packed CLI is installed) -- point it at a
clean temp directory distinct from any prefix an existing server was
installed into.
`install --force` is not needed for this replay, and must not be used just
to kill an unrelated apra-fleet server -- `--force` exists to stop the
server that owns the install being overwritten, not to clear the machine of
unrelated servers so a differently-scoped install can proceed. As long as
the data dir and install prefix are isolated from any running server, a
plain `apra-fleet install` (no `--force`) completes without the guard
firing.
</doc>
<doc title="Update" desc="Keeping Fleet and its members updated.">
# Self-Update Command
## What it does
`apra-fleet update` checks GitHub for the latest stable release and, if a newer
version exists, downloads its installer and runs it -- no manual download step
needed.
## Flags
| Command | Effect |
|---------|--------|
| `apra-fleet update` | Check for and install the latest stable release. |
| `apra-fleet update --check` | Report whether a newer release exists, without installing. |
| `apra-fleet update --help` | Show usage. |
## Behaviour
`apra-fleet update` with no flags:
1. Prints `Checking for updates...`.
2. Fetches release metadata from
`https://api.github.com/repos/Apra-Labs/apra-fleet/releases/latest` with a
5-second timeout. On a non-OK response it prints
`Error: Could not check for updates (Status: <code>)` and stops.
3. Compares the release tag against the installed version (the `_<hash>` build
suffix is stripped first). If the tag is a pre-release (`-alpha`, `-beta`,
`-rc`) or is not newer than the installed version, it prints an
"up to date" message and stops.
4. Selects the installer asset for the current platform:
| Platform | Asset |
|----------|-------|
| Windows x64 | `apra-fleet-installer-win-x64.exe` |
| macOS ARM | `apra-fleet-installer-darwin-arm64` |
| Linux x64 | `apra-fleet-installer-linux-x64` |
If no matching asset is found it prints
`Error: Could not find installer for platform <platform>` and stops.
5. Prints `Updating to <tag> -- restarting...`, downloads the installer into the
system temp directory, and (on macOS/Linux) marks it executable.
6. Spawns the installer detached, then exits with status 0:
```
<installer> install --force --llm <provider> --skill <skill> --workflows <mode>
```
`--force` makes the installer stop the running apra-fleet server before
replacing the binary. The `--llm`, `--skill`, and `--workflows` values come
from `install-config.json` (see below).
Any unexpected error is caught and printed as `Error: Update failed -- <message>`.
## install-config.json
`update` reads `~/.apra-fleet/data/install-config.json` and uses the first
provider entry to recover the `--llm` provider, its `--skill` set, and its
persisted `workflowsMode` (`all` or `none`), so the update preserves the
original install configuration -- including refreshing built-in workflow
assets to the new version while leaving any user-authored workflows on disk
untouched. `workflowsMode` is optional for backward compatibility: an
`install-config.json` written before the workflow subsystem existed won't
have the field, so `update` defaults it to `all`, matching a fresh install.
If the file is missing or cannot be parsed, `update` prints a warning and
falls back to `--llm claude --skill all --workflows all`.
## Stopping and restarting the server
The installer is run with `--force`, so it stops the running apra-fleet server
(and its workers) before overwriting the binary. The server is not a daemon --
it starts again on demand the next time an LLM CLI connects to it (run `/mcp` in
Claude Code, or restart the CLI).
`update` spawns the installer detached and exits 0 immediately, so it prints
`Updating to <tag> -- restarting...` before the installer has actually finished.
To confirm the new version is in place, check afterwards:
```
apra-fleet --version
```
## Notes
- `apra-fleet update` always installs when a newer stable release exists. Use
`apra-fleet update --check` first if you only want to see whether one is
available.
- The 5-second timeout covers the release-metadata check only; the installer
download itself is not time-bounded.
- The installer overwrites the binary in place with no `.bak`. To roll back,
download the previous release's installer and run it with `--force`.
</doc>
<doc title="SSH Setup" desc="Enabling SSH access on a remote machine.">
<!-- llm-context: Step-by-step guide for enabling SSH on remote machines so they can be registered as fleet members. Covers Linux (OpenSSH), macOS, and Windows (OpenSSH Server). Consult when a user can't connect to a remote member or needs to set up SSH for the first time. -->
<!-- keywords: SSH, setup, OpenSSH, Linux, macOS, Windows, sshd, firewall, key auth, remote member, connectivity -->
<!-- see-also: ../README.md (member registration after SSH is ready), adr-oob-password.md (password handling) -->
# SSH Server Setup for Fleet Members
Enable SSH on remote machines so they can be registered with `register_member`.
---
## Windows
Windows is the most involved. Run all commands in **PowerShell as Administrator**.
### 1. Install OpenSSH Server
```powershell
# Check if already installed
Get-WindowsCapability -Online | Where-Object Name -like 'OpenSSH.Server*'
# Install it
Add-WindowsCapability -Online -Name OpenSSH.Server~~~~0.0.1.0
```
Or: **Settings > Apps > Optional Features > Add a feature > OpenSSH Server**.
### 2. Start sshd and enable on boot
```powershell
Start-Service sshd
Set-Service -Name sshd -StartupType Automatic
```
### 3. Firewall rule
The installer usually creates this, but verify:
```powershell
# Check
Get-NetFirewallRule -Name *ssh*
# Create if missing
New-NetFirewallRule -Name sshd -DisplayName 'OpenSSH Server (sshd)' -Enabled True -Direction Inbound -Protocol TCP -Action Allow -LocalPort 22
```
### 4. Admin user gotcha
If your user is in the **Administrators** group, SSH ignores `~/.ssh/authorized_keys`. Keys must go in:
```
C:\ProgramData\ssh\administrators_authorized_keys
```
And the file needs restricted permissions:
```powershell
# Add your public key
Add-Content C:\ProgramData\ssh\administrators_authorized_keys "ssh-ed25519 AAAA... you@host"
# Fix permissions (must be owned by SYSTEM/Administrators only)
icacls C:\ProgramData\ssh\administrators_authorized_keys /inheritance:r /grant "SYSTEM:(F)" /grant "Administrators:(F)"
```
### 5. Default shell
Windows OpenSSH runs commands through whatever `HKLM:\SOFTWARE\OpenSSH`'s
`DefaultShell` value is set to (`cmd.exe` when unset). Fleet probes each
Windows member at registration and records the shell it proved
(`gitbash`, `pwsh7`, or `powershell5`) -- see
[windows-shell-selection.md](windows-shell-selection.md). If you want Fleet to
send Git-Bash-dialect commands, set the SSH `DefaultShell` to `bash.exe`, or
pass `shell` explicitly to `register_member` / `update_member`:
```powershell
New-ItemProperty -Path "HKLM:\SOFTWARE\OpenSSH" -Name DefaultShell `
-Value "C:\Program Files\Git\bin\bash.exe" -PropertyType String -Force
```
### 6. Admin vs non-admin
Fleet operations (registration, status checks, auth provisioning,
`execute_prompt`) do **not** require admin privileges. Prefer a non-admin SSH
account: Windows OpenSSH runs with full admin privileges -- UAC bypassed
entirely -- when the SSH user is in the Administrators group, so every Fleet
command would execute elevated.
Provider CLI install and update also work without admin. The Claude CLI, for
example, installs per-user to `C:\Users\<username>\.local\bin\claude.exe`; the
installer warns that `.local\bin` is not on PATH, and the fleet server
prepends it automatically before running provider commands.
### 7. Verify
From another machine:
```bash
ssh user@windows-host "echo ok"
```
If it connects but immediately closes, the default shell is likely
misconfigured (see step 5).
---
## macOS
### 1. Enable Remote Login
**System Settings > General > Sharing > Remote Login** (toggle on).
Or via terminal:
```bash
sudo systemsetup -setremotelogin on
```
### 2. Verify
```bash
ssh user@mac-host "echo ok"
```
---
## Linux (Ubuntu/Debian)
### 1. Install and start
```bash
sudo apt install openssh-server
sudo systemctl enable --now ssh
```
### 2. Verify
```bash
ssh user@linux-host "echo ok"
```
---
## Linux (RHEL/Fedora)
```bash
sudo dnf install openssh-server
sudo systemctl enable --now sshd
sudo systemctl status sshd
```
---
## Jetson / Embedded Linux
Usually pre-installed. Just confirm it's running:
```bash
sudo systemctl status sshd
```
If not running:
```bash
sudo systemctl enable --now sshd
```
---
## Troubleshooting
| Symptom | Cause | Fix |
|---|---|---|
| **Connection refused** | sshd not running | Start the service (see above) |
| **Permission denied** | Wrong password or key not deployed | Check `~/.ssh/authorized_keys` on target; on Windows admin users, check `administrators_authorized_keys` (see above) |
| **`All configured authentication methods failed`** | Wrong credentials, or password auth disabled | macOS uses the account's login password, not a separate SSH password, and its usernames are case-sensitive (confirm with `whoami` on the target). If the server is key-only, set `PasswordAuthentication yes` in `/etc/ssh/sshd_config` |
| **Connection timed out** | Firewall blocking port 22 | Add inbound rule for TCP/22: Linux `sudo ufw allow ssh` or `sudo firewall-cmd --add-service=ssh --permanent && sudo firewall-cmd --reload`; Windows, see the firewall rule above. macOS opens SSH automatically when Remote Login is enabled |
| **Windows: key auth not working for admin user** | Keys in `~/.ssh/authorized_keys` are ignored for admin accounts | Move keys to `C:\ProgramData\ssh\administrators_authorized_keys` and fix permissions |
| **Host key verification failed** | Host key changed (reinstall, new machine) | `ssh-keygen -R <host>` to clear old key |
</doc>
<doc title="Out-of-Band Auth" desc="Securely managing credentials and passwords for fleet members.">
# OOB Auth -- Terminal Mechanism and SSH/Headless Fallback
Covers the design of the out-of-band credential collection system and its SSH/headless fallback.
---
## Background: Why OOB Auth Exists
Credentials (passwords, API keys, confirmation prompts) must not pass through the LLM -- the model must never see plaintext secrets. The OOB (out-of-band) mechanism collects credentials in a separate UI context, passes them over a local socket, and delivers them to the fleet server without them appearing in the prompt stream.
---
## Unix Domain Socket (UDS) Architecture
The fleet server creates a socket at `~/.apra-fleet/data/auth.sock` (Linux/macOS) or a Windows named pipe equivalent. This is a filesystem object -- any process running as the same user on the same machine can reach it.
**Flow:**
1. A tool requiring a credential calls `collectOobInput()` (in `src/services/auth-socket.ts`).
2. `collectOobInput` registers a pending auth request with a 10-minute TTL via `createPendingAuth()`.
3. It calls `launchAuthTerminal()` to open a terminal window running `apra-fleet secret --set <memberName>`.
4. The launched process prompts the user, reads input with masked display (LLM cannot see it), and sends the value over the UDS as a JSON message.
5. `collectOobInput` **blocks** -- `waitForPassword()` awaits a Promise that resolves only when the credential arrives over the socket (or a cancellation/timeout fires). The call does not return early with a "Waiting..." status.
6. On receipt, the credential is consumed from the pending store and returned to the caller. The tool call then completes with a success message of the form `[OK] NAME stored [session/persistent]. Use {{secure.NAME}} in commands.`
**Key property:** The UDS socket is a filesystem object -- no GUI or display server is required to write to it. Any process on the machine, including one launched in a second SSH terminal, can deliver credentials.
---
## Display Detection
### Problem
`launchAuthTerminal` attempted GUI terminal emulators in order (`gnome-terminal -> xterm -> x-terminal-emulator`) on Linux. On SSH sessions:
- `which gnome-terminal` succeeds even when `$DISPLAY` is unset (the binary exists but can't connect).
- Spawn succeeds -> process exits immediately -> **"[FAIL] Password entry cancelled"** error fires.
- The error implies the user cancelled, not that the environment is headless.
The same issue on Windows: `start /wait cmd.exe` opens a window on the physical console, invisible to the SSH user.
### Solution
Two environment-variable checks added to `auth-socket.ts`:
```typescript
// Returns true when X11 or Wayland display is available
export function hasGraphicalDisplay(): boolean {
return Boolean(process.env.DISPLAY || process.env.WAYLAND_DISPLAY);
}
// Returns true when running on an interactive Windows desktop session
// SSH sessions and service contexts have SESSIONNAME !== 'Console'
export function hasInteractiveDesktop(): boolean {
return process.env.SESSIONNAME === 'Console';
}
```
`launchAuthTerminal` checks these **before** attempting any terminal emulator:
- **Linux, `$DISPLAY` and `$WAYLAND_DISPLAY` both unset:** Skip all GUI terminal emulators. Return actionable fallback message.
- **Windows, `SESSIONNAME !== 'Console'`:** Skip `cmd.exe start /wait`. Return actionable fallback message.
- **GUI desktop (display available):** Unchanged -- auto-launches terminal as before.
### Why check env vars rather than probing the socket
Probing (attempting a spawn and checking exit code) is what caused the misleading error in the first place. Env var checks are fast, zero-side-effect, and accurate for the cases that matter: X11/Wayland forwarding sets `$DISPLAY`, and Windows service contexts have a distinct `SESSIONNAME`.
Edge case accepted: X11 forwarding where `$DISPLAY` is set but the forwarded display is unreachable. This is acceptable -- if the terminal fails to launch, existing fallback logic catches it, and a user with X11 forwarding active almost certainly has a working display.
**Windows OOB window close:** Closing the OOB terminal window on Windows now returns immediately with a cancellation message. Previously, closing the window caused the tool call to hang for up to 5 minutes waiting for a password that would never arrive. The fix attaches a close-signal handler so that window dismissal resolves the pending Promise with a cancellation error immediately.
---
## The `! apra-fleet secret --set <name>` Pattern
On headless environments, the fallback message instructs the user to run:
```
! apra-fleet secret --set <actual-member-name>
```
The `!` prefix is the Claude Code "run in shell" operator -- it executes the command in the user's terminal without passing it to the LLM. This is the **single-terminal approach**: the user does not need to open a second window; they run the secret collection command inline in the same Claude Code session.
The message includes the **actual member name** (not a placeholder). The member name is available at the point `launchAuthTerminal` is called -- it is passed as the `memberName` parameter.
**Full fallback message text (Linux headless, credential-collection mode):**
```
fallback:No graphical display detected (SSH or headless session).
Run this in a separate terminal to provide the credential:
! apra-fleet secret --set <memberName>
Alternatively, pre-store the value with credential_store_set and reference it as {{secure.NAME}} in the credential field.
```
**Full fallback message text (Linux headless, egress-confirm mode):**
```
fallback:No graphical display detected (SSH or headless session).
Run this in a separate terminal to confirm:
! apra-fleet secret --confirm <memberName>
Alternatively, pre-store the value with credential_store_set and reference it as {{secure.NAME}} in the credential field.
```
The `fallback:` prefix is a protocol marker consumed by `collectOobInput` to distinguish the fallback path from a successful terminal launch. It is stripped before the message reaches the user.
---
## Fallback: Second Terminal
When the `!` operator isn't available or the user is in a non-Claude Code context, the fallback instruction is to open a second terminal and run `apra-fleet secret --set <memberName>` there. Because the UDS socket is a filesystem object, the second terminal's `apra-fleet secret --set` process connects to the same socket and delivers the credential to the waiting fleet server -- no GUI required.
---
## Re-entrancy and Stale State
If a terminal launch fails (fallback path), `collectOobInput` cleans up the pending auth state:
- Clears the `passwordWaiters` entry for the member
- Without this, `hasPendingAuth()` returns `true` on the next call, the re-entrant path skips `launchAuthTerminal`, and the call hangs waiting for a credential that will never arrive.
This cleanup ensures retries always start fresh.
</doc>
<doc title="Git Authentication" desc="How git authentication works across fleet members.">
<!-- llm-context: Design doc for fleet's git authentication system -- scoped token provisioning via GitHub Apps, PATs, Bitbucket, and Azure DevOps. Read when a user asks how to give members git access, how tokens are scoped, or how credentials are managed. -->
<!-- keywords: git auth, GitHub App, PAT, Bitbucket, Azure DevOps, token, scope, provision, revoke, credential, push, pull, clone -->
<!-- see-also: ../README.md (step-by-step git auth setup), design-vcs-auth-onboarding.md (onboarding flow) -->
# Design: Git Authentication for Fleet Members
## Problem
Fleet members need git access (clone, push, force-push, issue management) across multiple git hosts (GitHub, Azure DevOps, Bitbucket, GitLab). Without a standardized provisioning path, git credentials land on members ad hoc and cannot be scoped per member role.
Key requirements:
- **Multi-host**: Same abstraction across GitHub, Azure DevOps, Bitbucket, GitLab, self-hosted
- **Scoped permissions**: Read-only members shouldn't be able to push; dev members shouldn't force-push to main
- **Short-lived tokens**: Compromised member = limited blast radius
- **Zero user plumbing**: Users declare intent ("this member needs read access"), fleet handles the rest
- **Audit trail**: Every token mint logged with member name, scope, timestamp
## Design
### User-Facing Config
Members declare git access in their registration or member config:
```yaml
members:
code-analyst:
host: 192.168.1.13
work_folder: /Users/akhil/git/ApraPipes
git_access: read
git_repos: [Apra-Labs/ApraPipes]
feature-dev:
host: 192.168.1.13
work_folder: /Users/akhil/git/ApraPipes
git_access: push
git_repos: [Apra-Labs/ApraPipes]
release-bot:
host: 192.168.1.14
work_folder: /home/deploy/releases
git_access: admin
git_repos: ["*"]
project-mgr:
host: local
work_folder: C:\akhil\project-tracking
git_access: issues
git_repos: [Apra-Labs/ApraPipes, Apra-Labs/apra-lic-mgr]
```
### Access Levels
| Level | Git operations | Non-git |
|---|---|---|
| `read` | clone, pull, fetch, blame, log | - |
| `push` | read + push to branches (branch protection blocks main/force-push) | - |
| `admin` | read + push + force-push + tags + releases | CI/CD triggers |
| `issues` | - (no code access) | issues, PRs, projects, comments |
| `full` | admin + issues | Everything |
### VCS provider resolution
A member's VCS provider (`github` | `bitbucket` | `azure-devops` | `none`)
determines which credential backend `provision_vcs_auth` targets and which
PR-shaped command `fleet-sprint` builds for it. `register_member` accepts an
explicit `vcs_provider`; when omitted, registration reads the member's git
`origin` remote (best effort) and maps its host to a provider
(`github.com` -> `github`, `bitbucket.org` -> `bitbucket`, `dev.azure.com` /
`*.visualstudio.com` -> `azure-devops`). A GitHub Enterprise host has no
fixed domain and is never auto-detected -- register those members with an
explicit `vcs_provider`.
Auto-detection commonly fails at registration time, because the ordinary
flow is register-then-clone: the member's work folder has no git repo yet,
so there is no `origin` to read. Registration still succeeds in that case
(a missing VCS provider does not block onboarding), but emits a loud warning
that the member cannot push or open a PR until one is set. There are three
ways to resolve it after the fact: call `provision_vcs_auth` with an explicit
`provider` (this also records `vcsProvider` as a side effect of provisioning
credentials); call `update_member` with `vcs_provider` set, to record the
provider directly without provisioning credentials; or rely on
`fleet-sprint`'s dispatch-time fallback, which re-attempts the same
remote-based detection once a git remote exists and self-heals the
registry entry automatically. Re-registering the same folder path is
rejected as a duplicate registration, so it is never the remedy for a wrong
or missing auto-detect.
### Backend: GitHub App Token Minting
For GitHub-hosted repos, use a **GitHub App** installed on the org.
```
+---------------------------------------------+
| apra-fleet-app (GitHub App) |
| Installed on: the org |
| App private key stored on PM/master |
| |
| Max permissions (app-level): |
| - contents: write |
| - issues: write |
| - pull_requests: write |
| - actions: write |
| - administration: write |
+--------------+------------------------------+
|
PM mints scoped tokens per member at runtime:
|
+--> code-analyst: { contents: read, repos: [ApraPipes] }
+--> feature-dev: { contents: write, repos: [ApraPipes] }
+--> release-bot: { contents: write, admin: write, repos: [*] }
+--> project-mgr: { issues: write, pull_requests: write }
```
**Token minting flow:**
```typescript
// Using @octokit/app
import { App } from "@octokit/app";
const app = new App({
appId: FLEET_GITHUB_APP_ID,
privateKey: FLEET_GITHUB_APP_KEY,
});
async function mintGitToken(agent: Agent): Promise<string> {
const octokit = await app.getInstallationOctokit(installationId);
const { token } = await octokit.request(
"POST /app/installations/{installation_id}/access_tokens",
{
installation_id: installationId,
repositories: agent.git_repos, // scoped to specific repos
permissions: mapAccessLevel(agent.git_access), // scoped permissions
}
);
return token; // valid for 1 hour
}
function mapAccessLevel(level: string): Record<string, string> {
switch (level) {
case "read": return { contents: "read" };
case "push": return { contents: "write" };
case "admin": return { contents: "write", administration: "write", actions: "write" };
case "issues": return { issues: "write", pull_requests: "write" };
case "full": return { contents: "write", administration: "write", issues: "write", pull_requests: "write", actions: "write" };
}
}
```
**Credential deployment to member:**
```typescript
async function provisionGitAuth(agent: Agent): Promise<void> {
const token = await mintGitToken(agent);
// Configure git credential helper on the member
await agent.executeCommand(
`git config --global credential.helper '!f() { echo "password=${token}"; }; f'`
);
// Or more robustly, write a credential helper script
await agent.executeCommand(`cat > ~/.fleet-git-credential << 'EOF'
#!/bin/sh
echo "protocol=https"
echo "host=github.com"
echo "username=x-access-token"
echo "password=${token}"
EOF
chmod +x ~/.fleet-git-credential
git config --global credential.helper ~/.fleet-git-credential`);
}
```
### Backend: Azure DevOps
Use an **Azure AD App Registration** (Service Principal):
```typescript
// Using @azure/identity + azure-devops-node-api
const credential = new ClientSecretCredential(tenantId, clientId, clientSecret);
const token = await credential.getToken("499b84ac-1321-427f-aa17-267ca6975798/.default");
// Deploy to member as PAT-style credential
await agent.executeCommand(
`git config --global credential.helper '!f() { echo "password=${token.token}"; }; f'`
);
```
### Backend: Bitbucket
Use a **Bitbucket OAuth Consumer** or **Repository Access Token**:
- OAuth Consumer: org-level, token minting via client_credentials grant
- Repository Access Token: per-repo, created via Bitbucket API, scoped permissions
### Backend: Self-hosted / GitLab
- GitLab: **Project Access Tokens** or **Group Access Tokens** via API
- Self-hosted: SSH keys (fallback -- no token API available)
### Token Lifecycle
```
Member startup / first git operation
|
v
PM mints scoped token (1hr TTL)
|
v
Deploy credential to member via execute_command
|
v
Member uses git normally (clone/push/etc)
|
v
Token nearing expiry? Auto-refresh before next git operation
|
v
Member deregistered? Token expires naturally (1hr max)
```
### MCP Tool Interface
The tool is `provision_vcs_auth` (`src/tools/provision-vcs-auth.ts`), with
`revoke_vcs_auth` as its counterpart. Its input carries a member identifier
plus a `provider` (`github` | `bitbucket` | `azure-devops`), an optional
credential `label` and `scope_url`, and a per-provider credential group:
- **GitHub**: `github_mode` (`github-app` | `pat`), `token`, and the
`git_access` / `repos` overrides for the GitHub App path.
- **Bitbucket**: `email`, `api_token`, `workspace`.
- **Azure DevOps**: `org_url`, `pat`, `pat_expires_at`.
Secret-bearing fields accept a `{{secure.NAME}}` token, resolved from the
credential store server-side so no secret passes through a model's context.
### Structured provisioning responses
`provision_vcs_auth`, `provision_auth` (the LLM-credential counterpart) and
`member_reservation` all return a structured MCP result (an `ok`/`failed`
discriminator, a machine-readable `reason` code, and tool-specific fields)
rather than emoji-prefixed prose. This matters for any orchestrator-side
consumer: `reason` values include benign-but-not-fully-successful outcomes
(for example `provision_auth`'s `skipped_local_member`) that still report
`ok: true`, so a consumer that only checks `ok` cannot distinguish "actually
provisioned" from "correctly skipped." Always branch on `reason` before
falling back to the generic `ok`/`failed` split. Any caller that stubs these
tools in a test must stub the structured shape (`{ text, structuredContent }`),
not the old string return -- a stub still shaped like the old prose return
passes type-checking (both are just objects) but throws at the first
`.structuredContent.ok` read, and if that call site is wrapped in a
best-effort `catch`, the throw is silently swallowed and the mismatch never
surfaces as a test failure.
### Server-side credential handoff (`vcs_credential_exec`)
Before this tool existed, the only way an orchestrator-side caller could run
a credential-requiring git/VCS command was to first learn the token itself:
dispatch the deployed git-credential-helper as a member command and parse
`password=<token>` out of the captured stdout. Even dispatched with a
silent flag, the plaintext token still round-trips through a command result
the orchestrator process reads.
`vcs_credential_exec` (`src/tools/vcs-credential-exec.ts`) removes that
round-trip. The caller sends a command containing one of two literal
placeholders where the credential belongs, and the server performs the
whole handoff in one call:
- `{{vcs_token}}` -- a **bare** reference, never inside the caller's own
quotes. The substituted value arrives already shell-escaped and fully
quoted for the member's shell, the same convention `execute_command`'s
`{{secure.NAME}}` tokens use.
- `{{vcs_token_inline}}` -- for a placement **inside the caller's own single
quotes** (e.g. `'Authorization: Bearer {{vcs_token_inline}}'`). The
substituted value here is the bare interior escaping only, not the fully
quoted form -- using `{{vcs_token}}` in that position would double-escape
the token and produce a broken command. Callers pick whichever placeholder
matches the syntactic position the token needs to land in.
The handoff itself:
1. Runs the member's deployed credential helper through `strategy.execCommand`
-- that output is consumed in-process and is never part of this tool's
result.
2. Substitutes `{{vcs_token}}` with the token, shell-escaped per
`isPosixShell(agentOs, agentShell)` -- never assumed to be POSIX.
3. Dispatches the substituted command.
4. Redacts any occurrence of the token from stdout/stderr before returning,
the same defense `execute-command.ts`'s output redaction applies.
The tool refuses any command containing **neither** placeholder, so it
cannot degrade into a second, unguarded `execute_command`. The plaintext
token appears in no field of any result an orchestrator-side caller can
read. `readMemberVcsCredentialToken` (the old prose-scraping path that
dispatched the credential helper as a plain member command and parsed the
token out of its stdout) has been retired from production entirely -- every
caller now routes through `vcs_credential_exec`. `buildCredentialReadCommand`
(the per-shell command string builder `readMemberVcsCredentialToken` used to
call) is kept: it is still `vcs_credential_exec`'s own in-process building
block for constructing the credential-read step of the handoff, so it stays
exported and tested even though nothing outside this tool calls it as a
standalone helper anymore.
An OS/shell combination with no credential-read implementation is a hard
failure, not a soft warning: the tool reports `reason: "unsupported_member_os"`
and refuses to run rather than falling through to a credential-less dispatch
that would silently fail downstream for a less diagnosable reason. Dialect
selection for the substituted command routes through the same
`isPosixShell(agentOs, agentShell)` helper the rest of the codebase uses, so
a Windows member on Git Bash correctly gets POSIX-style escaping rather than
being assumed cmd/PowerShell just because its OS is Windows.
Redaction must cover every exit path the substituted command can take, not
just its successful stdout/stderr -- a dispatch failure that throws (the
`dispatch_failed` catch branch) can still carry the token verbatim inside
the thrown error's own message (many transport layers echo the failed
command back into the error text). A redaction pass applied only to the
success path leaves that throw path as an unguarded leak of exactly the
secret the tool exists to protect.
### Security Properties
| Property | How it's achieved |
|---|---|
| **Least privilege** | Token scoped to declared repos + access level |
| **Short-lived** | 1hr tokens, auto-refreshed |
| **Auditable** | Token minting logged with member, scope, timestamp |
| **Revocable** | Remove member = token expires naturally; revoke app installation for emergency |
| **No secrets on members** | Members never see the app private key, only short-lived tokens |
| **Compromised member** | Max 1hr window, scoped to declared repos only |
### Comparison with Alternatives
| | SSH Keys | PATs | GitHub App (this design) |
|---|---|---|---|
| Per-member scoping | No | Manual | Automatic |
| Token lifetime | Forever | Days-years | 1 hour |
| Multi-host | Same key everywhere | Different per host | Abstracted |
| User effort | Generate + register keys | Generate + distribute tokens | Declare `git_access: push` |
| Revocation | Manual key removal | Manual token revocation | Auto-expires |
| Audit | SSH logs | None built-in | Full mint log |
## Delivery pieces
1. **GitHub App setup** -- create app, install on org, store private key in fleet config (`setup_git_app`, `src/services/github-app.ts`)
2. **`provision_vcs_auth` tool** -- mints scoped token, deploys credential to member; `revoke_vcs_auth` tears it down
3. **Auto-provisioning** -- mint token on member startup or first git operation
4. **Auto-refresh** -- check token expiry before git operations, refresh if needed
5. **Multi-host backends** -- GitHub, Bitbucket, and Azure DevOps adapters behind the same `provision_vcs_auth` interface (`src/services/vcs/`); GitLab is not implemented
6. **Member config** -- `git_access` and `git_repos` fields on `register_member` / `update_member`
</doc>
<doc title="Provider Guide" desc="Choosing an LLM provider -- strengths, role recommendations, and gotchas.">
<!-- llm-context: User-facing guide for choosing an LLM provider in apra-fleet. Consult when a user asks which provider to use for a role (PM, doer, reviewer), what each provider is good at, or what limitations to expect. For CLI flags, credential paths, and integration internals, see provider-matrix.md. -->
<!-- keywords: provider, Claude, Antigravity, Codex, Copilot, choose, role, PM, doer, reviewer, gotchas, limitations, context window, max_turns, OAuth -->
<!-- see-also: ../README.md (provider setup instructions), provider-matrix.md (full CLI and integration reference) -->
# Choosing an LLM Provider
Fleet supports Claude, Antigravity (agy), Codex, Copilot, and OpenCode. Members can run different providers and mix them freely within a single fleet.
## Provider strengths
- **Claude** - Balanced coding and reasoning; fine-grained per-tool permissions via `settings.local.json`.
- **Antigravity** - High-performance Gemini-based agentic CLI; supports large context windows, background tasks, and native beads task tracking.
- **Codex** - Structured-output enforcement via `--output-schema`; native subagent parallelism for concurrent subtasks with less orchestration overhead.
- **Copilot** - Multi-model marketplace (Claude + GPT families in one CLI); auto-compaction keeps sessions running indefinitely.
## Recommended provider by role
| Role | Recommended | Why |
|------|-------------|-----|
| PM (orchestrator) | Claude Code or Antigravity (agy) | Both plan and orchestrate well - both support planning, background tasks, and premium models (e.g., Opus / premium-tier). |
| Doer | Any provider | Sonnet, Antigravity, Codex, Copilot - mix freely. |
| Reviewer | Premium-tier models | Catches subtle issues smaller models miss. |
## Gotchas worth knowing
- **`max_turns` is Claude-only.** On Codex, Copilot, and Antigravity, use `timeout_s` instead to bound execution time.
- **Copilot needs a paid GitHub Copilot subscription** (Pro, Business, or Enterprise) and has the smallest context window (64K). It is best suited for smaller, focused tasks.
## Mixing providers in one fleet
Every member runs its own LLM backend, and they collaborate across vendors. Put
a Claude doer with an Antigravity reviewer, or the reverse -- the reviewer's model
disagrees with the doer's by construction, so it catches issues a same-model
review would wave through.
A fleet that has run in production:
```
pm-1 Opus 4.7 orchestrator
doer-1 Sonnet 4.6 feature work
doer-2 Antigravity large-context tasks
reviewer Opus 4.7 final review
```
**OpenCode and local models.** OpenCode works with any OpenAI-compatible
endpoint (Ollama, vLLM, etc.), so it is the provider for self-hosted models.
The model endpoint is the user's responsibility -- Fleet installs the CLI and
agents but does not provision or manage the inference server. Configure the
provider and base URL in `opencode.json`; see
[opencode-getting-started.md](opencode-getting-started.md) for details.
Because OpenCode members can run any model, model tiers (cheap / standard /
premium) are set per member at registration via `model_tiers` in
`register_member`. A single-model entry fills all three tiers.
**Registering a member from a shell.** `apra-fleet register-member --name
<name> --path <folder> [options]` is a shell-drivable equivalent of the
`register_member` MCP tool, for contexts that can run shell commands but
cannot make MCP tool calls (scripted setup, CI, an agent role without MCP
tool access). It shares the exact same validation and registration logic as
the MCP tool -- both converge on one underlying registration function, so
the two entry points can never drift apart. Run `apra-fleet register-member
--help` for the full flag reference.
---
To override which model each tier resolves to on a per-provider basis, see
[Customizing model tier mapping](install.md#customizing-model-tier-mapping).
---
Extending Fleet's provider support, or need the full CLI / integration detail? See [docs/provider-matrix.md](provider-matrix.md).
</doc>
<doc title="Cloud Compute" desc="AWS and cloud compute integration for fleet members.">
<!-- llm-context: This guide covers apra-fleet's AWS EC2 integration -- auto start/stop, GPU-aware idle detection, long-running tasks, cost tracking, and custom workload detection. Consult when a user asks about cloud instances, GPU workloads, cost management, or task monitoring. -->
<!-- keywords: AWS, EC2, cloud, GPU, nvidia-smi, idle detection, auto stop, cost tracking, long-running task, monitor_task, cloud_control -->
<!-- see-also: ../README.md (general setup), architecture.md (how fleet manages members) -->
# Cloud Compute Guide
## 1. Overview
Cloud compute extends apra-fleet with full EC2 lifecycle management:
- **Auto-start**: stopped EC2 instances start automatically when a tool is called
- **Idle auto-stop**: instances stop themselves after a configurable period of inactivity
- **Long-running tasks**: background task wrapper survives SSH disconnects, auto-retries on crash, keeps idle manager from stopping the instance while work is running
- **GPU monitoring**: `fleet_status`, `member_detail`, and `monitor_task` report live GPU utilization via `nvidia-smi`
- **Cost visibility**: uptime and estimated on-demand cost shown in status output
---
## 2. Architecture
```
PM (Claude)
|
| MCP calls
v
apra-fleet MCP server (this machine)
| |
| SSH (execCommand) | AWS CLI
v v
EC2 instance AWS EC2 API
(cloud member) (start/stop/describe)
```
The MCP server runs on your local machine. It talks to EC2 via the AWS CLI and to the instance via SSH. The idle manager runs inside the server process, polling every 60 seconds.
---
## 3. AWS Setup
### IAM permissions required
The AWS identity (user or role) running the server needs:
```json
{
"Effect": "Allow",
"Action": [
"ec2:StartInstances",
"ec2:StopInstances",
"ec2:DescribeInstances"
],
"Resource": "*"
}
```
### AWS CLI configuration
Install the AWS CLI and configure credentials:
```bash
aws configure # default profile
aws configure --profile apra # named profile (use cloud_profile param)
```
Verify access:
```bash
aws ec2 describe-instances --instance-ids i-0abc123def456789a --region us-east-1
```
---
## 4. Registering a Cloud Member
```
register_member(
friendly_name = "gpu-trainer",
work_folder = "/home/ubuntu/training",
member_type = "remote",
username = "ubuntu",
cloud_provider = "aws",
cloud_instance_id = "i-0abc123def456789a",
cloud_region = "us-east-1",
cloud_profile = "apra", # optional AWS CLI profile
key_path = "/home/you/.ssh/gpu-trainer.pem",
cloud_idle_timeout_min = 30, # auto-stop after 30min idle
)
```
**Parameter notes:**
| Parameter | Required | Notes |
|---|---|---|
| `cloud_provider` | yes | Only `"aws"` supported |
| `cloud_instance_id` | yes | EC2 instance ID, e.g. `i-0abc...` |
| `cloud_region` | no | Default: `us-east-1` |
| `cloud_profile` | no | AWS CLI named profile |
| `key_path` | yes | Path to SSH private key on this machine; also sets the SSH `key_path` for the member |
| `cloud_idle_timeout_min` | no | Default: 30. Per-agent idle timeout in minutes. |
The instance does **not** need to be running at registration time. The server will start it on first use.
---
## 5. Auto-Start
When `execute_command`, `execute_prompt`, or `send_files` is called on a cloud member, `ensureCloudReady()` runs first:
1. Calls `aws ec2 describe-instances` to get current state
2. **stopped** -> calls `aws ec2 start-instances`, waits for `running` state
3. **stopping** -> waits for `stopped`, then starts
4. **pending** -> waits for `running`
5. **running** -> verifies public IP is current, updates registry if changed
6. **terminated / shutting-down** -> throws error (cannot be used)
After the instance is running:
- Polls SSH port (TCP connect) every 2 seconds, up to 60 seconds
- Re-provisions Claude OAuth credentials (`provision_llm_auth`) -- F5
- Re-mints GitHub App tokens if the member has git repos configured -- F5
The returned agent object has the fresh public IP. All subsequent SSH calls use it.
---
## 6. Idle Auto-Stop
The idle manager runs in the background and checks all cloud members every 60 seconds.
**Stop conditions (all must be true):**
1. Member has been idle longer than `cloud_idle_timeout_min` (per-agent) or the global fallback
2. Instance is currently `running`
3. No GPU compute processes detected (`nvidia-smi` shows no active jobs)
4. No fleet or other Claude processes running in the work folder
**Timer reset:** Every successful tool call (`execute_command`, etc.) calls `touchAgent()`, which resets the idle timer for that member.
**Server restart persistence (R-9):** On startup, the idle manager pre-loads `lastActivity` from each member's `lastUsed` timestamp in the registry. A member that was active 5 minutes before a server restart will not be stopped until the full timeout expires from that last-used time.
**Safe default:** If activity cannot be determined (SSH unreachable, nvidia-smi error), the stop is deferred. Unknown = don't stop.
---
## 7. Long-Running Tasks
For GPU training jobs or other tasks that outlast an SSH session, use `long_running=true`:
```
execute_command(
member_id = "<gpu-trainer-id>",
command = "python train.py --epochs 100 --data /data/train",
long_running = true,
max_retries = 3,
restart_command = "python train.py --resume checkpoint.pt", # F1
)
```
**What happens:**
1. A bash wrapper script is generated and base64-encoded
2. The wrapper is decoded and written to `~/.fleet-tasks/<task_id>/run.sh` on the member
3. Launched with `nohup bash run.sh &` -- survives SSH disconnect
4. Returns immediately: `Task launched: task_id=task-<id>`
**Wrapper behavior:**
- Writes PID to `task.pid`, JSON status to `status.json`
- Background loop touches `~/.fleet-tasks/<task_id>/activity` every 5 minutes while running -- this prevents the idle manager from stopping the instance during active work (F3)
- On non-zero exit: retries up to `max_retries` times using `restart_command` (F1)
- `restart_command` is designed for checkpoint resume (different flags on retry)
- Falls back to `command` if `restart_command` not provided
- On completion or max retries: updates `status.json`, removes `task.pid`
**Checking progress:**
```
monitor_task(
member_id = "<gpu-trainer-id>",
task_id = "task-lx4k2z",
auto_stop = true, # stop instance automatically when task completes
)
```
Returns JSON:
```json
{
"taskId": "task-lx4k2z",
"status": "running", // running | completed | failed | retrying | unknown
"exitCode": null,
"retries": 0,
"started": "2026-03-18T10:00:00Z",
"updated": "2026-03-18T10:45:00Z",
"pidAlive": true,
"gpuUtilization": 87,
"logTail": "Epoch 45/100, loss=0.234..."
}
```
---
## 8. cloud_control Reference
Manual control over a cloud member's instance:
```
cloud_control(member_id="<id>", action="start") # start + wait for SSH ready
cloud_control(member_id="<id>", action="stop") # stop immediately (bypasses idle timer)
cloud_control(member_id="<id>", action="status") # show current state + cost
```
**Actions:**
| Action | Behaviour |
|---|---|
| `start` | Calls `ensureCloudReady` -- starts the instance, waits for SSH, re-provisions auth |
| `stop` | Calls `aws ec2 stop-instances` directly -- immediate, no idle check |
| `status` | Calls `getInstanceDetails` -- returns state, IP, instance type, uptime, estimated cost |
**`stop` vs idle auto-stop:** `cloud_control stop` bypasses all activity checks. Use it to forcefully stop an instance regardless of what's running. The idle manager's stop goes through GPU + process checks first.
---
## 9. Cost Estimation
`fleet_status` and `member_detail` show an estimated running cost based on:
- Instance type (from `aws ec2 describe-instances`)
- Uptime = `now - LaunchTime` (from the same API call)
- Hourly rate from a built-in lookup table (`src/services/cloud/cost.ts`)
**Supported families:** g4dn, g5, p3, p4d, t3, m5, c5 (us-east-1 on-demand pricing)
**Limitations:**
- Rates are hard-coded approximations. Actual AWS charges may differ due to spot pricing, savings plans, data transfer, EBS, etc.
- Instance types not in the table show `?` for cost.
- The lookup table is in `src/services/cloud/cost.ts` -- edit `HOURLY_RATES` to add custom types or update prices.
---
## 10. Supported Platforms
Cloud compute features are designed and tested for **Linux** EC2 instances (Ubuntu, Amazon Linux 2, Debian).
| Feature | Linux | macOS | Windows |
|---|---|---|---|
| GPU detection (`nvidia-smi`) | [OK] Full | [x] Not supported | [x] Not supported |
| Long-running task wrapper | [OK] Full | [!] Untested | [x] Not supported |
| Idle activity monitoring | [OK] Full | [!] Partial | [x] Not supported |
| Auto-start / auto-stop | [OK] Full | [OK] Full | [OK] Full |
| SSH connectivity | [OK] Full | [OK] Full | [!] Requires OpenSSH |
**Notes:**
- Registering a cloud member with a non-Linux OS will succeed but show a warning about unsupported features.
- The task wrapper script (`long_running=true`) uses `bash`, `nohup`, and POSIX shell utilities. It is not compatible with Windows Command Prompt or PowerShell.
- macOS GPU detection is not supported (`nvidia-smi` is not available on Apple Silicon or macOS in general).
- Only `aws` is supported as a `cloud_provider`. GCP and Azure support is planned.
</doc>
<doc title="Writing Skills" desc="How to write your own skill for use in the fleet.">
# Writing a skill
Fleet ships the `fleet` skill (the MCP tool reference) and the `pm` skill (the
Project Manager workflow), plus two small companion skills -
`auto-sprint-args` and `fleet-sprint-cli`. A skill is how you package a
*workflow* on top of Fleet so it can be invoked by name and reused. This page explains what a skill is and how to
build your own.
## What a skill is
A skill is a directory of Markdown that your AI coding agent loads as
instructions. It is not compiled code and there is no plugin API to implement --
a skill *describes* a workflow, and the agent carries it out using Fleet's MCP
tools.
The PM skill, for example, is Markdown that tells the agent how to plan a
sprint, dispatch a doer, run a reviewer, and raise a PR. Everything it does, it
does by calling Fleet tools like `register_member` and `execute_prompt`.
## Where skills live
Skills are installed into your provider's skills directory:
| Provider | Directory |
|----------|-----------|
| Claude | `~/.claude/skills/<name>/` |
| Antigravity (agy) | `~/.gemini/antigravity-cli/skills/<name>/` |
Fleet's installer writes `fleet/`, `pm/`, `auto-sprint-args/` and
`fleet-sprint-cli/` there. Your own skill is just another directory alongside
them.
## Anatomy
A skill directory contains one required file and any number of supporting ones:
```
my-skill/
SKILL.md <- required: the entry point
helper-notes.md <- optional: sub-documents the agent reads on demand
templates/ <- optional: files the skill sends to members
scripts/ <- optional: helper scripts
```
### SKILL.md
`SKILL.md` opens with YAML frontmatter, then the workflow body:
```markdown
---
name: my-skill
description: One sentence on what this skill does and when to use it.
note: This skill requires the 'fleet' skill to function.
---
# My Skill
You are a ... that ...
## Step 1
...
```
- `name` -- the skill's identifier; matches the directory name.
- `description` -- used to decide when the skill is relevant. Be specific.
- `note` -- optional; declare a dependency on the `fleet` skill if you call
Fleet MCP tools.
Keep `SKILL.md` focused. Push detail into sub-documents and reference them by
filename so the agent loads them only when needed -- this is how the `pm` skill
keeps `SKILL.md` short while `single-pair-sprint.md`, `doer-reviewer.md`, and
the `tpl-*.md` templates carry the depth.
## The tools a skill can use
A skill coordinates agents through Fleet's MCP tools. The most common:
| Tool | Use |
|------|-----|
| `register_member` | Add a machine or local workspace as a member. |
| `execute_prompt` | Run an AI prompt on a member. |
| `execute_command` | Run a shell command on a member (no tokens). |
| `send_files` / `receive_files` | Move files to and from a member. |
| `compose_permissions` | Generate provider-native permission config. |
| `fleet_status` / `member_detail` | Inspect member state. |
The `fleet` skill documents the full tool set. Your skill should activate the
`fleet` skill (via the `note` field) rather than re-documenting tools.
## Build your own
1. Create `~/.claude/skills/my-skill/SKILL.md` with the frontmatter above.
2. Write the workflow as numbered steps, in plain imperative prose. Reference
Fleet tools by name where the agent should call them.
3. Move long reference material into sibling `.md` files; link them by filename.
4. Test by invoking the skill in your AI coding agent and watching it run.
## Worked examples
The two skills in this repository are the best reference:
- [`skills/fleet/SKILL.md`](../skills/fleet/SKILL.md) -- the MCP tool reference
and member-management mechanics.
- [`packages/apra-fleet-se/apra-pm/skills/pm/SKILL.md`](../packages/apra-fleet-se/apra-pm/skills/pm/SKILL.md) -- a full multi-step workflow:
sprint variants, doer-reviewer pairing, templates, and lifecycle commands.
Read those alongside this page when designing your own.
</doc>
<doc title="Beads" desc="How Fleet uses Beads -- the bundled open-source issue tracker and the PM skill's persistent task database.">
<!-- llm-context: Reference for how Apra Fleet uses Beads, the bundled open-source issue tracker. Consult when a user asks about bd commands, task tracking, sprint epics, backlog management, or the PM skill's persistent task state. -->
<!-- keywords: Beads, bd, task, epic, sprint, backlog, pm, lifecycle, dependency, issue tracker, bd ready, bd create, bd close -->
<!-- see-also: ../README.md (PM skill overview), ../packages/apra-fleet-se/apra-pm/skills/pm/SKILL.md (PM skill reference), ../packages/apra-fleet-se/apra-pm/skills/pm/beads.md (internal PM Beads rules) -->
# Beads -- Fleet's Persistent Task Tracker
Beads is a bundled open-source local issue tracker installed alongside Fleet by
`apra-fleet install`. It provides the `bd` CLI and serves as the PM skill's
persistent task database across all sprints and sessions.
## What Beads Does
- **One central DB** -- the PM agent runs `bd init` once in its own working
directory. A single Beads database tracks all projects, all members, and all
sprints. Run `bd list --all --pretty` for a global view without reading files.
- **Epics and tasks** -- each sprint gets an epic; each PLAN.md task gets a child
task with priority, assignee, and dependency links (`bd dep add`).
- **Cross-session persistence** -- task state survives PM restarts. On session
start, run `bd ready` for an instant cross-sprint view of what is in flight.
- **Lifecycle hooks** -- the PM skill calls `bd` at every phase boundary: init,
plan approval, task dispatch, verify checkpoint, reviewer findings, and cleanup.
## Common Commands
| Command | What it does |
|---------|-------------|
| `bd ready` | Show all unblocked, open tasks across all sprints |
| `bd list --all --pretty` | Full tree: all projects, members, tasks, status |
| `bd create "<title>" -p <pri> --parent <epic-id> --assignee <member>` | Create a task |
| `bd create "<title>" --id <explicit-id>` then `bd update <explicit-id> --parent <epic-id>` | Create a task with a pre-decided id under a parent -- `bd create` rejects a call that combines `--id` and `--parent` in the same invocation, so an explicit id and a parent link are always two separate calls, never one |
| `bd show <id> --json` | Full detail for one task |
| `bd update <id> --status in_progress --assignee <member>` | Mark a task in progress |
| `bd close <id>` | Mark a task complete (idempotent) |
| `bd reopen <id>` | Reopen a prematurely closed task |
| `bd dep add <task-id> <blocks-id>` | Declare a dependency |
| `bd search "<term>" --status all --json` | Search tasks; check before creating to avoid duplicates |
| `bd note <id> "<text>"` | Attach a note (e.g. PR URL at sprint close) |
Priority values: `0` = highest, `3` = lowest (backlog).
## PM Lifecycle Hooks
The PM skill integrates Beads at these points in every sprint:
| PM command | Beads action |
|-----------|-------------|
| `/pm init` | `bd init` (idempotent); create sprint epic; record epic ID in status.md |
| `/pm plan` (after approval) | `bd create` one task per PLAN.md item; `bd dep add` for dependencies |
| `/pm start` / task dispatch | `bd update <id> --status in_progress --assignee <member>` |
| VERIFY checkpoint done | `bd close <id>` |
| Reviewer CHANGES NEEDED | `bd create` a task per HIGH finding |
| `/pm cleanup` | `bd close <epic-id>`; `bd note <epic-id> "PR: <url>"` |
## Backlog and Deferred Items
```bash
# Defer an item at low priority
bd create "<description>" -p 3 --parent <epic-id>
# Show full tree including deferred items
bd list --all --pretty
```
## Cross-Sprint Dependencies
Block sprint B on sprint A:
```bash
bd dep add <sprint-B-epic-id> <sprint-A-epic-id>
```
`bd ready` will not surface sprint B tasks until sprint A closes.
## Recovery After a PM Restart
Session crash? Run `bd list --all --pretty`. The PM sees every member's state
across every active project without reading a single file. Then `bd show <id>`
for full context on any item.
## Multi-member sync (Dolt-backed clones)
Each member's beads DB is an embedded Dolt clone. `apra-fleet install` also
provisions a portable Dolt CLI binary alongside `bd` (verified with a version
check at install time) -- this is a hard prerequisite for any sync path that
needs to inspect or resolve a Dolt clone directly, not merely an optional
extra. When more than one member's clone needs to reconcile against a shared
Dolt remote (for example, an autonomous multi-member sprint whose members are
genuinely independent checkouts rather than one shared workspace), reads and
writes are bracketed with a pull before and a push after so no member's clone
observes stale state for long, and every cross-member push is serialized
through a single coordinating authority -- a Dolt clone can hard-conflict on a
concurrent same-row write, and one unresolved conflict wedges that clone's
sync entirely, so multi-writer coordination is a correctness requirement, not
a performance nicety. See `packages/apra-fleet-se/docs/architecture.md`'s
"Dolt sync discipline" section for the full mechanism and its conflict
recovery ladder.
---
See [the PM skill](../packages/apra-fleet-se/apra-pm/skills/pm/SKILL.md) for the full PM skill reference.
</doc>
<doc title="FAQ" desc="Frequently asked questions about setup, troubleshooting, and the doer-reviewer loop.">
# Frequently Asked Questions
<!-- llm-context: This file points to the authoritative FAQ maintained as GitHub Discussions. Do NOT answer FAQ-type questions from cached knowledge -- fetch the linked discussion for the current, maintainer-verified answer. The discussion index at the link below is the single source of truth. -->
<!-- keywords: install, setup, single machine, local, provider, Claude, Antigravity, Codex, Copilot, doer, reviewer, token, cost, security, password, credential, A2A, agent2agent, FAQ, questions -->
<!-- see-also: ../README.md (getting started), architecture.md (how it works), provider-guide.md (choosing a provider) -->
> **For AI agents:** The FAQ is maintained as GitHub Discussions -- one discussion per question, with maintainer-verified answers. To answer a user's question: browse the index below, find the matching discussion, and fetch it for the authoritative answer. Do not paraphrase from this file -- follow the link.
All questions and answers are maintained at:
**[FAQ Index -- GitHub Discussions #127](https://github.com/Apra-Labs/apra-fleet/discussions/127)**
---
Want to explore the codebase or ask questions about how Fleet works internally?
The **[Fleet DeepWiki](https://deepwiki.com/Apra-Labs/apra-fleet)** has an AI assistant trained on the Fleet source code.
Some useful starting points:
- [How does execute_prompt work end-to-end?](https://deepwiki.com/Apra-Labs/apra-fleet/4.2-work-execution-tools)
- [How do I register and manage fleet members?](https://deepwiki.com/Apra-Labs/apra-fleet/4.1-member-lifecycle-tools)
- [What is the doer-reviewer workflow?](https://deepwiki.com/Apra-Labs/apra-fleet/7.1-sprint-lifecycle-and-doer-reviewer-loop)
---
Topics covered:
- **Getting started** -- installation, device requirements, provider support
- **Understanding members and workflows** -- icons, status line, doer-reviewer setup, folder separation
- **Capabilities and use cases** -- scope beyond software dev, credential security, custom skills
- **Ecosystem and protocols** -- relationship to Google's A2A protocol
- **Advanced / operations** -- token usage, crash recovery
---
**Related docs:** [Readme](../README.md) | [Architecture](architecture.md) | [Cloud Compute](cloud-compute.md) | [Provider Guide](provider-guide.md)
</doc>
<doc title="Troubleshooting" desc="Common symptoms and fixes.">
# Troubleshooting
Common symptoms and how to resolve them. If something here does not match what
you see, search [GitHub Issues](https://github.com/Apra-Labs/apra-fleet/issues)
or ask in [Discussions](https://github.com/Apra-Labs/apra-fleet/discussions).
## Members
**Member shows as offline**
- Check the machine is reachable: `ping <ip>`.
- For remote members, verify SSH directly: `ssh user@host "echo ok"`.
- If SSH works but the member is still offline, re-provision auth: ask Fleet to
"Provision auth for `<member>`".
**Empty response from a member**
Usually an expired auth token. Ask Fleet to "Provision auth for `<member>`".
For VCS tokens specifically, re-run `provision_vcs_auth`.
If it is the member's *first* dispatch after registration and the CLI exited
with code 0 and no output at all, suspect workspace trust rather than auth:
`execute_prompt` classifies an exit-0/empty-stdout dispatch against a
never-trusted workspace as `workspace_not_trusted`, seeds trust, and retries
once. See "Permission granted but still denied on Claude" below.
**`Claude CLI auth check failed -- you may need to run provision_llm_auth` during registration**
Normal for a new member. `register_member` still succeeds; run
`provision_llm_auth` for that member to finish setting up authentication.
Before provisioning: for the default OAuth flow, log in locally first (`/login`
in a Claude Code session, or `claude auth login`) -- `provision_llm_auth` copies
your credentials to the member. For the API-key flow, pass the key as the
`api_key` parameter. The tool checks token expiry before deploying; an expired
access token with a live refresh token still deploys, and the member's CLI
refreshes on first use.
**Auth error (401 / 403)**
- GitHub App tokens: re-mint with `provision_vcs_auth`.
- Bitbucket / Azure DevOps: the token likely expired -- get a fresh one, then
re-provision and retry. See the `auth-*.md` references in the fleet skill.
**Member blew past a checkpoint**
Check what actually happened on the member:
ask Fleet to run `cat progress.json` on it.
**Long-running background task fails to launch on a Windows member**
`execute_command`'s `long_running` mode on Windows launches the task
detached via `Invoke-CimMethod Win32_Process.Create` (WMI). If this fails,
check that the WMI service (`Winmgmt`) is running on the member and that the
fleet SSH user has permission to create processes via WMI -- `monitor_task`
will otherwise report the task as immediately failed/missing rather than
running.
## Permissions
**Permission denied on a member**
Fleet can configure member permissions. Ask it to, for example, "Grant
`build-server` permission to run `npm install`". Under the hood this runs
`compose_permissions`, which writes provider-native config:
| Provider | Config location |
|----------|-----------------|
| Claude | `.claude/settings.local.json` |
| Codex | `.codex/config.toml` (approval mode) |
| Copilot | `.github/copilot/settings.local.json` |
**Permission granted but still denied on Claude**
Claude Code only honors `.claude/settings.local.json` permissions once the
project folder is a **trusted workspace**. If the member's work folder has
never been opened and trusted in Claude Code directly, the permissions Fleet
writes there are inert. Open the folder in Claude Code once and accept the
trust prompt, then retry.
**A deploy step is blocked at the permission pre-check even though similar commands are allowed**
Check the allowlist for the *exact* subcommand the deploy step invokes, not
just the binary name -- an allowlist entry scoped to one subcommand (e.g.
`Bash(<tool> start)`) does not cover a different subcommand of the same
binary (e.g. `Bash(<tool> run ...)`) even though both start the same
long-running process. Read the deploy runbook's own Permissions section for
the full list of required command prefixes and diff it against
`.claude/settings.json` / `.claude/settings.local.json` before assuming the
underlying operation itself is unsafe or needs a workaround -- add the
missing prefix and re-trigger.
## Timeouts
A dispatch can end in two distinct ways:
- **Inactivity timeout (`timeout_s`)** -- fires when no stdout/stderr output
arrives for N seconds (default 300s / 5 min). It is transport-level, so it
applies to every member and provider. The usual cause is a build or test
runner that buffers output (`npm test`, `vitest`, `cargo build`) and stays
silent for long stretches even while working. Fix: raise `timeout_s` to
600-1200 for build/test dispatches.
- **Total timeout (`max_total_s`)** -- fires after N seconds of wall-clock time
regardless of output. Use it as a hard ceiling on long jobs, alongside
`timeout_s` when you want both a silence guard and a wall-clock cap.
## Credentials
**A token or password appeared in command output**
Store the secret with `credential_store_set`, then reference it as
`{{secure.NAME}}` in `execute_command`. Fleet redacts it to `[REDACTED:NAME]`
before the LLM ever sees the output. See
[docs/features/oob-auth.md](features/oob-auth.md).
**Rotate a credential without re-provisioning**
Run `credential_store_delete name=<NAME>` then `credential_store_set
name=<NAME>`. The new value is picked up immediately on the next
`execute_command` that references `{{secure.NAME}}`.
## Git
**Cannot push workflow files or merge PRs from a member**
Minted VCS tokens may lack CI/CD permissions. Run those operations from your
main AI coding session instead -- it has your full git credentials. See
[docs/design-git-auth.md](design-git-auth.md).
**`git push` from a Windows member hangs or fails silently with no auth prompt**
If the member's git config has `credential.helper=manager` (Git Credential
Manager), the helper tries to open an interactive prompt that cannot appear in
a headless session, so the push stalls or fails without a useful error. Use a
non-interactive credential source instead: `gh auth setup-git` (GitHub) or the
token minted by `provision_vcs_auth`, which writes a scoped credential entry
that needs no prompt.
**`bd init` errors "already initialized" on a second run**
`bd init` is not idempotent. Any script or playbook that bootstraps beads must
check for an existing `.beads/` directory (or `bd` reporting a database) before
calling it, rather than treating the error as a failure.
## Build & native dependencies
**`npm ci`/`npm install` fails compiling a native module (e.g. `better-sqlite3`) via `node-gyp`**
This is a toolchain/environment incompatibility, not a code defect -- do not
patch the deploy or build scripts to work around it. On macOS it shows up as
Xcode Command Line Tools' `libc++` headers rejecting newer C++20 constructs
(`concept`, `requires`, `output_iterator_tag`/`contiguous_iterator_tag`) that
the Node header set expects, once the Node major version and the installed
CLT/node-gyp versions drift out of the combination that was actually tested.
Resolve by aligning the toolchain, not the source tree: update Xcode Command
Line Tools (`xcode-select --install` / reinstall), or build against a Node
version known to match the installed CLT, or update `node-gyp` itself. Check
the npm debug log path printed in the failure output for the exact compiler
error before assuming this is the cause.
## Stuck agents
**A member is stuck after a session reset**
Escalate the model tier (cheap -> standard -> premium) and retry. If it is still
stuck, the task likely needs a human decision -- inspect `progress.json` and
intervene directly.
## Logs
For unexplained behavior -- missing output, silent failure, unexpected results
-- check the server logs:
```
$APRA_FLEET_DATA_DIR/logs/fleet-<pid>.log
```
These are JSON lines. Filter by member or by tool with `jq`:
```bash
jq 'select(.member_id == "<uuid>")' fleet-<pid>.log
jq 'select(.tag == "<tool>")' fleet-<pid>.log
```
The **Fleet Logs** section of the fleet skill's `SKILL.md` has the full field
reference and more `jq` examples.
## Contributor gotchas
Failure modes that look like success when working on this repo.
**A `scripts/` entrypoint guard never fires on Windows**
Comparing `import.meta.url` against a hand-built `"file://" + process.argv[1]`
string never matches
on Windows: `process.argv[1]` is a native path (`C:\path\to\s.mjs`), so the
hand-built string has two slashes after the scheme, while `import.meta.url` is
a properly encoded URL with three slashes and forward slashes throughout. The
comparison is always false, `main()` never runs, and the script exits 0 having
verified nothing -- indistinguishable from a real pass. Use
`pathToFileURL(process.argv[1]).href` instead. Coverage for this class of
defect must spawn the script as a real CLI process; an in-process `import()`
does not exercise the guard at all.
**`execFileSync('bd', [...])` fails with `spawnSync bd ENOENT` on Windows**
The globally installed `bd` resolves on PATH to npm's `bd.cmd`/`bd.ps1` shims,
which `CreateProcess` cannot exec directly. Adding `{ shell: true }` "fixes" it
but reintroduces command injection, since Node joins the command and every
array argument into one unquoted command line. The repo's shared helper
(`scripts/lib/exec-bd.mjs` and its supervisor twin) instead parses the shim for
the underlying `bin/bd.js` path and spawns
`execFileSync(process.execPath, [scriptPath, ...args])` -- no shell involved.
Route new `bd` call sites through that helper rather than reimplementing.
**A green root test run can still miss a whole workspace**
A workspace whose tests run under a different runner than the root (e.g.
`node --test` alongside a root `vitest run`) is invisible to the root test
command unless its script is explicitly chained in. Verify the root gate
actually executes every workspace's suite before treating it as a gate.
**When spawning an interactive CLI from Node, close stdin**
`child_process.exec()` leaves the child's stdin connected to the parent. A CLI
that checks for a TTY (such as `claude -p`) then waits forever for input that
never arrives. Call `child.stdin?.end()` immediately after the spawn.
</doc>
<doc title="Roadmap" desc="What is planned next for Apra Fleet.">
# Roadmap
This roadmap is grounded in the repo's actual git history and issue tracker
(beads), not aspiration. "Shipped" means it is on `main`. The forward
sections are a projection of where the current trajectory leads; priorities
shift based on what the fleet itself surfaces while building this codebase.
For what is actively being worked on right now, read the beads backlog
(`bd ready`) rather than this file.
Have an idea? [Open a feature request](https://github.com/Apra-Labs/apra-fleet/issues/new/choose).
---
## What's shipped
### The fleet-sprint engine and the workflow platform
- **`fleet-sprint`: an autonomous multi-agent sprint engine** -- plan ->
develop -> review -> harvest cycles run by planner/doer/reviewer role
agents against a real git repo, with beads (`bd`) as the task DAG and
Dolt as the sync backend. Renamed from `auto-sprint` to end the
confusion with Claude Code's unrelated `/auto-sprint` script; source
lives in `packages/apra-fleet-se/fleet-sprint/`, and this repository is
itself built by it (see the dashboard recording in README.md).
- **`apra-fleet workflow <name>`: a general-purpose workflow runner** --
the `packages/apra-fleet-workflow` engine gives workflows typed errors,
budget enforcement, resumable/replayable runs, and cooperative `/stop`
cancellation. fleet-sprint is the first workflow shipped on it.
- **`apra-fleet-se`: an always-on multi-sprint supervisor (preview)** --
runs several sprints concurrently against a shared fleet with a
member + issue-scope reservation ledger, a PID-liveness watchdog with
restart re-adoption, orchestrator-bracketed git+Dolt sync with a
scripted-first conflict ladder, and a sprint-stack dashboard (running
sprints, history, backlog tree) on one reverse-proxied port. Still gated
"preview" until the full end-to-end supervisor smoke cycle passes
cleanly.
- **Sprint reliability hardening** --
credential-auth self-heal via `provision_vcs_auth`, orphaned-CLI
recovery, PID-liveness "lease of life" so a flaky channel no longer
produces false empty-response failures, a stall detector that watches
all transcript activity (not just chat) and kills confirmed stalls,
and dispatch-vs-sync failure separation so a Dolt/git push hiccup no
longer burns a full LLM re-dispatch.
- **MCP transport defaults to streamable HTTP** -- one long-lived
`apra-fleet run` server on `localhost:7523/mcp` shared by every
provider, replacing per-session stdio subprocesses; `--stdio` remains
as an alias.
- **apra-pm "comes home"** -- the `vendor/apra-pm` submodule is gone;
apra-pm is a package-local dependency at `packages/apra-fleet-se/apra-pm`,
which removed a whole class of silent submodule drift in CI/e2e/packaging.
- **Deterministic e2e harness** -- fleet setup/teardown is a script, not
an LLM improvisation; checkpoint consolidation is deterministic; and the
turn-budget, git-auth, and log-collection flakes are fixed.
### Providers and members
- **OpenCode provider** -- full adapter (NDJSON parseResponse, session
management, permissions/auth), per-member `model_tiers` with
dispatch-time resolution and validation against available models,
GLM-4.5-Air premium default, e2e suites on GitHub-hosted runners, and
`docs/opencode-getting-started.md`. This is the door to local/self-hosted
OpenAI-compatible models.
- **Antigravity (agy) provider maturation** -- `--agent` flag dispatch,
session-resume fixes, safety rationalization (`docs/agy-safety-rationalization.md`).
- **Role-agent file installation, including remote members** --
`apra-fleet install` writes planner/doer/reviewer/plan-reviewer agent
definitions into each provider's agents directory, and `update_member`
provisions them to remote members too.
- **Member categories and tags** -- `category` plus up to 10 `tags` on
register/update, tag-filtered `list_members`, and tag-driven
`compose_permissions` profile merging. This shipped what the old
roadmap called "member groups".
- **No-LLM members** -- `llm_provider: none` for machines that only run
commands or host services (GPU nodes, relays).
- **Live member activity viewer** -- `apra-fleet watch` streams what every
member is doing.
- **CLI ergonomics** -- `install` is the default action and `run` starts
the MCP server; bare Claude model aliases instead of pinned dated IDs;
npm packaging and the SEA binary coexist on every supported platform.
### The knowledge layer
- **Knowledge Bank MCP tools** -- `kb_session_prime`, `kb_capture`,
`kb_query`, `kb_harvest`, `kb_promote`, `kb_export` and the rest of the
`kb_*` family, scoped per repo and opt-in per repo (`kb_setup`). See
[docs/knowledge-layer.md](docs/knowledge-layer.md).
- **Code-intelligence provider abstraction** -- `code_graph`,
`code_impact`, `code_query`, `code_context` route through a pluggable
provider, selected per member via `codeIntelProvider`, with repo-scoped
`kb_harvest` firing automatically after `execute_prompt`.
---
## Near-term (next few weeks)
The near-term is dominated by finishing what the supervisor opened, not by
new surface area.
- **Pass the supervisor end-to-end smoke gate and drop the "preview"
label.** The declared acceptance for the supervisor -- a full
plan-develop-review-harvest cycle through apra-fleet-se against a live
sandbox -- is not yet proven end to end. This is the single most
important open item in the repo.
- **Supervisor dashboard parity** so it is a strict superset of the
fleet-sprint viewer, and make apra-fleet-se the single supported entry
point for running sprints (the CLI stays as the low-level path).
- **Prove the knowledge layer's value with eval evidence** before turning
it on by default: merge only what paired eval sprints show is a
measurable win, behind the per-member/per-repo provider routing that
already has tests.
- **Windows parity as a standing theme.** The bug record (POSIX-only agent
checks, bd ENOENT spawns, pipefail on non-bash shells, silent Windows
SEA build failures) says cross-platform drift is the most common
regression class; expect continued small fixes plus regression guards
rather than one big effort.
- **One transform pipeline for role agents** across all providers, instead
of provider-specific handling of agent definitions.
## Mid-term (1-3 months)
Extrapolating from what the architecture is clearly reaching toward:
- **Hub-spoke cloud mode becomes usable.** The groundwork is already in
the tree -- `packages/fleet-api-contract`, `docs/hub-spoke-master-plan.md`,
the wire-protocol and hub-service-deployment docs, the identity model,
and the `apra-fleet join` / `apra-fleet spoke` CLI commands -- but spoke
mode is not end-to-end yet. With HTTP transport the default and the
supervisor a long-lived service, a hub that remote spokes attach to is
the natural next step, and it is the prerequisite for any hosted
offering.
- **Dashboard auth and RBAC.** `docs/dashboard-oauth-rbac-design.md`
exists for a reason: the moment the supervisor dashboard is the front
door to a shared, always-on service (and especially a hub), it needs
login and roles. Expect this to ride immediately behind hub-spoke.
- **A second real workflow on apra-fleet-workflow.** The engine was
explicitly built for more than fleet-sprint (see
`docs/authoring-workflows.md`). The credible proof that "any workflow"
is real is a shipped second workflow -- likely something operational
(release playbook automation or an e2e/integration runner) since those
already exist as semi-manual scripts in this repo.
- **Knowledge layer graduates from opt-in to default.** If the eval
results hold up, the trajectory is: code-intel provider on by default
for sprint members, repo-scoped harvest on session close, and then a
central/team KB server -- in that order, each behind evidence.
- **Cost governance surfaces in the dashboard.** The pieces exist
(`docs/cost-model.md`, `get_member_model_pricing`, per-turn cost
calculation); the missing piece is per-sprint/per-member cost rollups
where operators actually look -- the supervisor dashboard.
- **Sprint calibration data starts steering sprints.** The engine already
writes `sprint calibration` and `sprint-analysis` commits every cycle;
the obvious next step is feeding that history back into planning
(cycle-count estimates, model-tier selection, stall-timeout tuning)
instead of only recording it.
## Long-term (3-12+ months)
- **Hosted fleet / fleet-as-a-service on the hub-spoke foundation.** Once
spoke mode and dashboard RBAC exist, a managed hub is mostly an ops
problem, not an architecture problem. Multi-fleet federation
(hub-of-hubs) is the step after, and should stay speculative until at
least two independent hubs exist in practice.
- **Workflows beyond software engineering.** The README already markets
"any domain" (retail replenishment, logistics exceptions, intake
triage); making that true requires the mid-term items first: a proven
second workflow, workflow authoring docs that outsiders can follow, and
an extension/distribution story for workflows that are not baked into
the repo.
- **Enterprise governance: audit trail and policy.** The security
substrate is unusually strong already (OOB secrets, per-provider
permission composition, the permission-block-surfacing convention); the
missing enterprise piece is an immutable audit log of fleet operations
and secret usage, which becomes mandatory the moment a hosted hub has
more than one tenant.
- **Close the self-development loop fully.** The endgame the dogfooding
is pointing at: the supervisor runs continuously against this repo,
fleet-sprint files, fixes, reviews, and ships its own bugs with the
human as reviewer-of-last-resort, and the calibration/KB layers make
each sprint measurably cheaper than the last. Everything above --
supervisor smoke gate, knowledge layer, cost rollups, calibration
feedback -- is a component of that loop.
### Deliberately not on the roadmap
Items the evidence does not currently
support prioritizing: gbrain integration (superseded by the
code-intelligence abstraction), Playbooks as a separate feature (largely
subsumed by the workflow engine), Slack notifications and a Terraform
provider (no recent activity or demand signal in the tracker). They can
return if demand shows up.
---
## Contributing
Pick an item above, open an issue to discuss your approach, then submit a
PR. See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.
</doc>
<doc title="Contributing" desc="How to contribute code, docs, and skills.">
# Contributing to apra-fleet
Thank you for your interest in contributing! This document explains how to get involved.
## Reporting Bugs
Use the [Bug Report](https://github.com/Apra-Labs/apra-fleet/issues/new/choose) issue template on GitHub. Include as much detail as possible -- reproduction steps, environment info, and error output are especially helpful.
## Requesting Features
Use the [Feature Request](https://github.com/Apra-Labs/apra-fleet/issues/new/choose) issue template. Describe the problem you're trying to solve, your proposed solution, and any alternatives you've considered.
## Development Setup
**Prerequisites:** Node.js 22.16+, npm
```bash
git clone https://github.com/Apra-Labs/apra-fleet.git
cd apra-fleet
npm install
npm run build
```
`npm install` auto-installs the git pre-commit hook via the `prepare` script. To install manually, run `node scripts/install-hooks.mjs`. The hook lives at `.github/hooks/pre-commit`.
## Running Tests
```bash
npm test
```
For watch mode during development:
```bash
npm run test:watch
```
## Branch Naming
| Type | Pattern | Example |
|------|---------|---------|
| Feature | `feat/<short-description>` | `feat/ec2-support` |
| Bug fix | `fix/<short-description>` | `fix/ssh-timeout` |
| Docs | `docs/<short-description>` | `docs/contributing-guide` |
Always branch from `main`.
## Commit Message Convention
Use the [Conventional Commits](https://www.conventionalcommits.org/) format:
```
<type>(<scope>): <short summary>
```
Common types: `feat`, `fix`, `docs`, `chore`, `refactor`, `test`
Examples:
- `feat(members): add EC2 instance support`
- `fix(ssh): handle connection timeout gracefully`
- `docs: update contributing guide`
## Pull Request Process
1. Fork the repo and create your branch from `main`.
2. Make your changes, following the code style notes below.
3. Run `npm run build` and `npm test` -- both must pass.
4. Open a PR against `main` using the PR template.
5. A maintainer will review your PR. Address any feedback.
6. Once approved, a maintainer will merge it.
## Code Style
- **Language:** TypeScript. Match the style of surrounding code.
- **Formatting:** No enforced formatter currently -- keep indentation and style consistent with existing files.
- **No unnecessary abstractions:** Prefer simple, direct code over premature generalization.
- **Error handling:** Only handle errors at real system boundaries (user input, SSH, external APIs). Don't add fallbacks for scenarios that can't happen.
- **ASCII only:** No non-ASCII characters in committed files. Use `--` for em-dashes, `->` for arrows, `[OK]` for checkmarks.
## For AI Agents
If you are an AI agent (or a human using an AI agent) contributing to this project, this section covers the patterns and conventions that matter most.
### Dev-mode install
Build and install from source without touching the packaged binary:
```bash
npm run build && node dist/index.js install
```
This registers the MCP server from your local `dist/` build. Skill files are read from `skills/` on disk -- no rebuild needed to iterate on them.
### File map
| Path | What it contains |
|------|-----------------|
| `src/` | TypeScript source for the MCP server, CLI commands, and providers |
| `skills/fleet/` | Fleet skill -- tools for managing members, tasks, and files |
| `packages/apra-fleet-se/apra-pm/skills/pm/` | PM skill -- orchestration patterns, doer-reviewer loop, deploy flows |
| `packages/apra-fleet-se/apra-pm/agents/` | Role agent definitions (planner, doer, reviewer, deployer, ...) |
| `hooks/` | Shell hooks that run on Claude Code events (statusline, pre-push, etc.) |
| `CLAUDE.md` | Shared project context; the source AGENTS.md and AGY.md are generated from by `node scripts/sync-agent-docs.mjs` |
### Testing skill changes
Skills are Markdown files -- edits take effect immediately without a rebuild. After editing a skill under `skills/` or `packages/apra-fleet-se/apra-pm/skills/`:
1. Save the file.
2. In Claude Code, run `/mcp` to reload the MCP server.
3. The updated skill content is live.
Run `npm test` before committing to catch any regressions in the TypeScript layer.
### Doer-reviewer loop
The PM agent delegates tasks to doer members and assigns a separate reviewer. Code is never self-reviewed. When implementing multi-step work:
- All task state lives in the beads (`bd`) task DB -- there is no `PLAN.md` and no `progress.json`.
- The PM reads `bd ready` and hands the doer explicit bead ids, one task at a time.
- Each doer commits and closes its bead.
- A reviewer member inspects the diff before the PM proceeds.
### Sprint branch naming
| Type | Pattern | Example |
|------|---------|---------|
| Feature sprint | `feat/<desc>` | `feat/install-ux-and-docs` |
| Sprint (generic) | `sprint/<desc>` | `sprint/q2-hardening` |
Agent-driven work always happens on a sprint branch -- never directly on `main`.
## License
By contributing, you agree that your contributions will be licensed under the [Apache License 2.0](LICENSE) that covers this project.
</doc>
</docs>
</project>