git:20260808.5ce8d82 to git:20260824.d18190a

---

name: kelly-agent-observability

description: Busabase App-in-Skill dashboard visualizing a generated MOCK fleet of LLM agents running behind a shared AI gateway for a generic organization. Use when the user invokes $kelly-agent-observability or /kelly-agent-observability, wants to review agent health, call volume, latency, error rates, cost, or trace-level chain breaks across an agent fleet. This is a demo/reference dashboard over a generated mock fleet snapshot seeded into Busabase by a trusted script — never a live monitoring integration, never a real gateway or real agents.

metadata:

category: platform

tags:

- risk:sandbox

- surface:busabase

---

# Agent Fleet Observability Desk

## Overview

Kelly Agent Observability is a Busabase Cloud App-in-Skill. Its canonical

product surface is the AirApp in Busabase, not a separate local-data

product. The same Hono source supports an explicitly requested local preview

with OAuth connection bootstrap. It gives an operator a read-mostly

dashboard over a MOCK fleet of LLM agents running behind a shared AI gateway

for a generic organization: per-agent call volume, latency (p50/p95),

error/failure rate, cost, an hourly call-volume sparkline, and a Trace Detail

view that highlights exactly where a chain of tool calls broke.

This skill was never a live monitoring integration, even before this

conversion — there is no real gateway, no real agents, and no real telemetry

anywhere. The fleet snapshot (agents + metrics + traces) is generated by a

deterministic, seeded function and written into Busabase by a trusted

skill-root script (`scripts/generate_fleet_data.mjs`); the AirApp itself only

ever **reads** whatever was last generated there. The only human action is

narrow: append an "acknowledge" / "needs investigation" handoff note to an

agent or trace — always a brand-new row in the `handoffs` Base, never a field

update on the agent/trace's own record.

Default behavior is AirApp-first. Unless the user explicitly asks only for

explanation, ensure the mock fleet exists (run the seed script below if the

`agents` Base is empty) and give the user the clickable AirApp URL. Start

localhost only when local preview/debugging is explicitly requested; it uses

the same Busabase resources and never offers another data provider. Use

chat-only mode only when the user says "纯聊天", "chat only", "no UI", or

similar.

This is deliberately **generic and brand-free**: agent names are generic

archetypes (Booking Assistant, Support Triage, etc.) for a generic

organization ("your organization", "the fleet") — no real company or product

appears anywhere in the code, config, or seed data.

## Mandatory Dependencies

1. Read and follow `$kelly-app-skill-creator` for product behavior, visual

2. Read and follow `$busabase` for connection, target Space, node discovery,

ChangeRequests, review, and merge behavior.

3. Read and follow `$busabase-app-creator` for resource modeling, AirApp

runtime limits, security, validation, and deployment.

If a dependency is unavailable, preserve this skill's artifact and product

contracts, stop before the unavailable Busabase operation, and report the

exact missing dependency. Do not invent a second data backend.

## App UI Screenshots

<table>

<tr>

<td width="50%"><img src="assets/screenshots/overview.webp" alt="Fleet overview"></td>

<td width="50%"><img src="assets/screenshots/agent-health.webp" alt="Agent health table"></td>

</tr>

<tr>

<td><strong>Fleet Overview</strong><br>Total calls, total cost, degraded/critical/healthy agent counts, and a per-agent sparkline card grid.</td>

<td><strong>Agent Health</strong><br>Sortable table with call volume, p50/p95 latency, error rate, cost, and a status badge per agent.</td>

</tr>

<tr>

<td width="50%"><img src="assets/screenshots/trace-detail.webp" alt="Trace detail with chain break"></td>

<td width="50%"><img src="assets/screenshots/overview.zh-CN.webp" alt="Fleet overview in Chinese"></td>

</tr>

<tr>

<td><strong>Trace Detail</strong><br>Ordered step timeline for one trace; the step where the chain broke is visually flagged.</td>

<td><strong>Chinese UI</strong><br>Full zh-CN chrome via the language toggle or <code>lang=zh</code>.</td>

</tr>

</table>

## Boundary

- Read-mostly dashboard over a generated mock fleet. NEVER contact any real

agent gateway, LLM provider, or third-party API. There is no live

monitoring integration in this skill by design.

- Agents and traces are never created or edited by the AirApp; they enter

Busabase only through the trusted `scripts/generate_fleet_data.mjs` seed

script, the same way `kelly-portfolio-health`'s contracts enter through an

external sync process the app doesn't control.

- The only Base the AirApp itself ever writes to is `handoffs`, and only ever

by creating a brand-new row (an acknowledge / needs-investigation note) —

never a field update on an agent or trace record.

- Never reference any real company, brand, or production system. Agent names

are generic archetypes for a generic organization ("your organization",

"the fleet").

## Busabase Resources

Four Bases under one application Folder (`kelly-agent-observability`),

- `agents`: one row per mock agent archetype (8 rows) — identity plus the

latest rolled-up health metrics (status, call volume, p50/p95 latency,

error rate, cost) and a 48h hourly call/error series (JSON array). Written

only by the trusted generator script.

- `traces`: one row per mock trace (ordered tool-call chain), capped to fit

under the 100-record read limit (8 agents × 10 traces = 80 by default).

`steps` (JSON array) carries the ordered timeline; `broke-at-step-id` flags

exactly where a broken chain stopped. Written only by the trusted generator

script.

- `handoffs`: append-only human-in-the-loop log — an acknowledge /

needs-investigation note against one agent or trace. The only Base the

AirApp itself ever writes to.

- `settings`: sanitized fleet-generation metadata (schema version, seed,

generated-at), one row keyed by `kind`.

Resources provision lazily through an idempotent Busabase ChangeRequest the

first time the app runs in a Space; see `references/fleet-schema.md` for

exact field shapes.

## First Run And Onboarding

On invocation, check the `agents` Base. If it's empty, run the trusted seed

script to generate the mock fleet:

```bash

node skills/kelly-agent-observability/scripts/generate_fleet_data.mjs --apply

```

There are no credentials to collect — this skill never calls an external

system, so onboarding is just running the seed script once.

## Local App

Default behavior is AirApp-first — give the user the clickable AirApp URL.

requested.

Required app views (hash routes):

- `#/overview`: total calls (24h), total cost (today), degraded/critical/

healthy agent counts, and a per-agent card grid with a call-volume

sparkline.

- `#/agents`: sortable Agent Health table (call volume, p50/p95 latency,

error rate, cost, status badge).

- `#/agents/<agent_id>`: agent detail — metrics, sparkline, recent traces,

and a handoff form (acknowledge / needs investigation).

- `#/traces/<trace_id>`: trace detail — ordered step timeline with the

chain-break step visually flagged, and a handoff form.

- `#/handoffs`: history of recorded handoffs.

- `#/settings`: data provider and status-threshold summary.

## Demo Mode

- `?demo=1` opens a deterministic, fully offline mock fleet (8 agents, 48h of

hourly buckets, 16 traces per agent) for documentation and screenshots. It

never reads or writes Busabase and never claims a real connection; demo

handoff submissions only update the in-memory list already rendered.

- `?demo=agents` and `?demo=trace` select named mock scenes (agent health

table, a broken trace).

- `lang=en` or `lang=zh` forces UI chrome language for screenshots.

UI language: support English and Chinese chrome with `Auto` default.

## Fleet Generation Model

verbatim from the retired `lib/generate.ts`:

- **Per-agent profile** — a fixed "personality" (volume, latency base/jitter,

error tendency, cost per call) per agent archetype, so the fleet reads as

varied, not random noise.

- **Hourly buckets** — 48 hours of calls/errors per agent on a

business-hours-ish curve, seeded per agent.

- **Status** — `statusFor(error_rate_pct, p95_latency_ms)`: `critical` at

error rate ≥ 8% or p95 ≥ 8000ms, `degraded` at ≥ 3% or ≥ 4000ms, else

`healthy`.

- **Traces** — an ordered tool-call chain per trace; a trace can "break" mid

chain (`status: error`, `broke_at_step_id` set to the last step), with the

break probability tied to the agent's error tendency.

Same seed + same "now" + same `tracesPerAgent` always produces bit-identical

output — this is the property `scripts/generate_fleet_data.mjs` and the demo

provider both rely on.

## Handoffs

The one human action — acknowledge or flag "needs investigation" on an agent

or trace, with a free-text note — writes a brand-new row into the `handoffs`

Base via `busabase-sdk`, never a field update on the agent/trace record

itself:

- From a standalone local preview the write merges immediately (trusted

operator).

- From the deployed AirApp it creates a pending ChangeRequest for the

trusted process to merge, per the AirApp boundary in

`$busabase-app-creator`.

## Safety

- Never invent a real incident or real company data; this is illustrative

mock telemetry only.

- Keep all writes scoped to the `handoffs` Base. No external network calls

anywhere in this skill.

- Redact nothing sensitive is collected here — there are no credentials in

this skill.

## Useful Commands

```bash

node skills/kelly-agent-observability/scripts/generate_fleet_data.mjs --apply

```

## Execution reports

Re-read the active provider's decisions immediately before any approved execution. Record each concrete operation, target, status, timestamp, and error in the provider-backed execution report; keep app actions local-only.