git:20260910.e36f2e2 to git:20260926.4c4b67c

3 added, 3 removed. Audit A to A.

---

name: pilot-watch-alert-response

description: 'Use when Pilot is RESPONDING TO REAL-TIME CLUSTER EVENTS surfaced by `cub-scout watch` or `cub-scout bot`. Natural phrasings: "Pilot subscribes to the watch stream", "render verdicts on each drift event as they fire", "real-time acceptance kernel for cluster changes", "Pilot, decide on this drift.detected event", "what does Pilot say about the ownership.changed alert", "stream PASS/WATCH/ASK/BLOCK per event from cub-scout watch". Pilot consumes the `cub-scout watch` / `cub-scout bot` event stream (webhook or file sink, optionally with inline receipts via `--emit-receipt-on`) and renders a verdict per event, calling back into cub-scout (`explain`, `compare three-way`, `trace`) for context when needed. This is the ONLY event-driven skill in the pilot-* batch — others are query-driven. Do NOT load for: a single-resource pre-deploy gate (use pilot-cd-gate), a periodic fleet roll-up (use pilot-fleet-conformance), a drift-classification deep-dive (use pilot-patch-and-drift), a postmortem evidence pack (use pilot-incident-evidence), or any mutating action.'

phase: cross-cutting

- allowed-tools: Bash(./cub-scout watch *) Bash(cub-scout watch *) Bash(cub scout watch *) Bash(./cub-scout bot *) Bash(cub-scout bot *) Bash(cub scout bot *) Bash(./cub-scout explain *) Bash(cub-scout explain *) Bash(cub scout explain *) Bash(./cub-scout compare three-way *) Bash(cub-scout compare three-way *) Bash(cub scout compare three-way *) Bash(./cub-scout compare drift *) Bash(cub-scout compare drift *) Bash(cub scout compare drift *) Bash(./cub-scout compare source-truth *) Bash(cub-scout compare source-truth *) Bash(cub scout compare source-truth *) Bash(./cub-scout trace *) Bash(cub-scout trace *) Bash(cub scout trace *) Bash(./cub-scout receipt verify *) Bash(cub-scout receipt verify *) Bash(cub scout receipt verify *) Bash(./cub-scout receipt show *) Bash(cub-scout receipt show *) Bash(cub scout receipt show *) Bash(./cub-scout receipt validate *) Bash(cub-scout receipt validate *) Bash(cub scout receipt validate *) Bash(./cub-scout receipt list *) Bash(cub-scout receipt list *) Bash(cub scout receipt list *) Bash(./cub-scout doctor *) Bash(cub-scout doctor *) Bash(cub scout doctor *) Bash(./cub-scout map *) Bash(cub-scout map *) Bash(cub scout map *) Bash(kubectl get *) Bash(kubectl describe *) Bash(cub * get) Bash(cub * list) Bash(cub link list *) Bash(cub unit get *) Bash(argocd app get *) Bash(flux get *)

+ allowed-tools: Bash(./cub-scout explain *) Bash(cub-scout explain *) Bash(cub scout explain *) Bash(./cub-scout compare three-way *) Bash(cub-scout compare three-way *) Bash(cub scout compare three-way *) Bash(./cub-scout compare drift *) Bash(cub-scout compare drift *) Bash(cub scout compare drift *) Bash(./cub-scout compare source-truth *) Bash(cub-scout compare source-truth *) Bash(cub scout compare source-truth *) Bash(./cub-scout trace *) Bash(cub-scout trace *) Bash(cub scout trace *) Bash(./cub-scout receipt verify *) Bash(cub-scout receipt verify *) Bash(cub scout receipt verify *) Bash(./cub-scout receipt show *) Bash(cub-scout receipt show *) Bash(cub scout receipt show *) Bash(./cub-scout receipt validate *) Bash(cub-scout receipt validate *) Bash(cub scout receipt validate *) Bash(./cub-scout receipt list *) Bash(cub-scout receipt list *) Bash(cub scout receipt list *) Bash(./cub-scout doctor *) Bash(cub-scout doctor *) Bash(cub scout doctor *) Bash(./cub-scout map list *) Bash(./cub-scout map workloads *) Bash(./cub-scout map orphans *) Bash(cub-scout map list *) Bash(cub-scout map workloads *) Bash(cub-scout map orphans *) Bash(cub scout map list *) Bash(cub scout map workloads *) Bash(cub scout map orphans *) Bash(kubectl get *) Bash(kubectl describe *) Bash(cub space list *) Bash(cub unit get *) Bash(cub unit list *) Bash(cub link get *) Bash(cub link list *) Bash(cub view get *) Bash(cub view list *) Bash(cub unit-event list *) Bash(cub resource list *) Bash(cub release list *) Bash(cub changeset list *) Bash(argocd app get *) Bash(flux get *)

---

# pilot-watch-alert-response

The real-time event-driven scenario from **Pilot's** side. Pilot subscribes to `cub-scout watch` or `cub-scout bot` (which expose the same webhook / file event sinks) and, for each event, decides PASS / WATCH / ASK / BLOCK — calling back into cub-scout for context as needed. This is the **only event-driven skill in the pilot-* batch**; the other four are query-driven.

With the receipts v2 surface shipped (#463), Pilot can ask `cub-scout watch` or `cub-scout bot` to **inline a receipt** with each matching event via `--emit-receipt-on drift.detected,ownership.changed` or `--emit-receipt-on all`. That delivers a fingerprinted in-toto Statement v1 per event in the same JSONL line — Pilot reads the verdict + receipt in one shot, no synchronous call-back required for the common case.

## When to use

Explicit phrasings:

- "Pilot subscribes to the watch stream"

- "Render verdicts on each drift event as they fire"

- "Real-time acceptance kernel for cluster changes"

- "Pilot, decide on this `drift.detected` event"

- "What does Pilot say about the `ownership.changed` alert"

- "Stream PASS/WATCH/ASK/BLOCK per event from cub-scout watch"

- "Run the same stream from cub-scout bot in the cluster"

- "Pilot reads receipts inline from the watch event stream"

- "Trigger Pilot on any cub-scout drift event"

Implicit intents:

- The flow is **event-driven**, not query-driven (Pilot reacts to a stream, not a polling job)

- The decision needs to **fire fast** — a Pilot verdict per event, ideally inline

- The receipt-attached path (v2 `--emit-receipt-on`) is preferred when audit durability matters

- Backpressure is a real concern when events fire fast (e.g., ApplicationSet expansion)

## Do not load for

- A **single-resource pre-deploy gate** — [`pilot-cd-gate`](../pilot-cd-gate/SKILL.md)

- A **periodic fleet roll-up verdict** — [`pilot-fleet-conformance`](../pilot-fleet-conformance/SKILL.md)

- A **drift-classification deep-dive** for one resource — [`pilot-patch-and-drift`](../pilot-patch-and-drift/SKILL.md)

- A **postmortem evidence pack** — [`pilot-incident-evidence`](../pilot-incident-evidence/SKILL.md)

- Pure **observation** without a Pilot policy decision — [`scout-observe`](../scout-observe/SKILL.md) covers the `watch` surface from the operator side

- Any **mutating action**. Pilot's mutation path is `cub` / Argo / Flux / kubectl — never cub-scout.

## The Pilot ↔ cub-scout watch / bot loop

```

cub-scout watch -n prod \

--webhook https://pilot.acme/cub-scout-events \

--emit-receipt-on drift.detected,ownership.changed \

--output-file /var/log/cub-scout-watch.jsonl

│

│ ▼ JSONL event stream (one event per line)

│ {

│ "type": "drift.detected",

│ "timestamp": "2026-05-22T11:00:00Z",

│ "resource": {"kind": "Deployment", "name": "api", "namespace": "prod"},

│ "owner": {"type": "argo", "name": "payments-api"},

│ "severity": "warning",

│ "details": {"category": "DRIFT"},

│ "receipt": { in-toto Statement v1 with applied-matches-spec verdict }

│ }

▼

Pilot's event handler

│ fast path: the event already carries a receipt

│ │

│ └► Pilot reads receipt.predicate.verdict → render verdict

│

│ slow path: the event doesn't carry a receipt

│ │ (event type isn't in --emit-receipt-on, OR receipt-build failed)

│ │

│ ├──► cub-scout compare three-way <r> -n <ns> --format json

│ ├──► cub-scout explain <r> -n <ns> --presentation ai

│ └──► cub-scout receipt verify <r> -n <ns> --strategy <s> --out event-receipt.json

│

▼

Pilot's per-event verdict

│ PASS → log + continue

│ WATCH → alert + log + (optional) attach receipt to a ticket

│ ASK → page on-call; the proof gap is structured in the receipt

│ BLOCK → halt promotion / trigger Pilot's escalation policy

└────────────────────────────────────────────────────

```

## Event types (`cub-scout watch` / `cub-scout bot` + receipts v2)

`cub-scout watch` and `cub-scout bot` emit four event types today; `--emit-receipt-on` can attach receipts to any of them, with `--emit-receipt-batch-cap` bounding first-poll bursts:

|------------|---------------|----------------------|--------------------------|

| `resource.discovered` | First poll sees a resource cub-scout hadn't seen before | **Yes** — bounded by `--emit-receipt-batch-cap` to avoid initial burst floods. | Log; no verdict needed. |

| `ownership.changed` | A resource's ownership controller changed (Argo → Flux, Flux → kubectl, etc.) | **Yes** — `applied-matches-spec` auto-detected. Receipt verdict reflects whether the new owner has a resolved git anchor. | WATCH (often a transient state during migration); ASK if `applied-matches-spec` returns INCONCLUSIVE. |

| `drift.detected` | `compare three-way` finds a divergence on a resource cub-scout watched into | **Yes** — `applied-matches-spec` auto-detected. Receipt verdict reflects whether the drift is controller-drift (PASS / reconciling) or manual-edit (BLOCK). | Drives [`pilot-patch-and-drift`](../pilot-patch-and-drift/SKILL.md) shape: PASS = ACCEPT; WATCH = INVESTIGATE; BLOCK = REVERT/QUARANTINE. |

| `scan.finding` | A risk pattern matched on a resource (CVE, expiring cert, etc.) | **Yes** — bounded by `--emit-receipt-batch-cap` to avoid initial scan floods. | Severity-driven; route to security/SRE escalation policy. |

The `--emit-receipt-on all` sugar accepts every known event type for forward-compat. Receipt-build failures are non-fatal — the underlying event still emits, just without the `receipt` key (uses `omitempty`; consumers should check key presence, not null-ness).

```python

# Idiomatic Python consumer

for line in stream:

event = json.loads(line)

if "receipt" in event: # key-present check, NOT `is not None`

verdict = pilot.decide_from_receipt(event["receipt"])

else:

verdict = pilot.fallback_decide(event) # slow path

handle(verdict)

```

## Wire shape

A `drift.detected` event with an attached receipt (one JSONL line, pretty-printed here):

```json

{

"type": "drift.detected",

"timestamp": "2026-05-22T11:00:00Z",

"resource": {

"kind": "Deployment",

"name": "payments-api",

"namespace": "prod"

"owner": {

"type": "argo",

"name": "payments-api"

"severity": "warning",

"details": {

"category": "DRIFT"

"receipt": {

"_type": "https://in-toto.io/Statement/v1",

"subject": [

{

"name": "k8s-live://apps/v1/Deployment/prod/payments-api",

"digest": {"sha256": "abc..."}

}

"predicateType": "https://cub-scout.dev/receipt/v1",

"predicate": {

"version": "v1",

"predicateName": "applied-matches-spec",

"verdict": "BLOCK",

"claim": "applied matches spec at apps/prod/payments-api",

"scope": {"kind": "Deployment", "name": "payments-api", "namespace": "prod"},

"evidence": {

"attribution": {"cause": "manual-edit", "managerHint": "kubectl-edit"},

"gitSource": {"repoUrl": "https://github.com/org/payments", "revision": "abc123", "path": "apps/prod/payments-api"}

"omissions": [],

"inputAttestations": [],

"nextSteps": [

{

"actionType": "read-only",

"reason": "live state was last written by a kubectl-* tool; the GitOps loop was bypassed since the controller's last apply",

"nextCommand": "kubectl get --show-managed-fields",

"nextSurface": "kubectl"

}

"verifier": {"tool": "cub-scout", "version": "v2.2.1"},

"verifiedAt": "2026-05-22T11:00:00Z",

"fingerprint": "sha256:1a2b3c..."

}

```

Notes on the wire shape (verified against `pkg/agent/receipt.go`):

- `predicate.version` is the literal string `"v1"` (`PredicateVersion`), not `"1"`.

- The `nextSteps` array is populated for the manual-edit BLOCK branch — `EvaluateAppliedMatchesSpec` appends one `read-only` next step pointing at `kubectl get --show-managed-fields`. (`nextSteps` uses `omitempty` so other branches that produce zero next steps omit the key entirely.)

- `omissions: []` and `inputAttestations: []` are explicit arrays (never null). The full receipt's fingerprint covers both.

Pilot reads `receipt.predicate.verdict` directly: `BLOCK` → halt; `WATCH` → alert; `PASS` → log + continue; `INCONCLUSIVE` → ASK. Receipt verdicts are restricted to those four — `ASK` is a source-truth `status` value (and maps to receipt `WATCH` when wrapped, per `pkg/agent/receipt_predicates.go:396-407`), not a receipt verdict.

## Step-by-step

### Step 1 — confirm the watch stream is shaped for Pilot

```bash

# Pre-flight: doctor + status

$ cub-scout doctor

$ cub-scout status

# Confirm the event taxonomy Pilot expects

$ cub-scout watch --help | grep -A 3 '\-\-emit-receipt-on'

```

If standalone (no ConfigHub auth), the receipts in the stream omit the `confighub-unit://` subject and record an `OmissionConfigHubUnitSubject` entry — structural honesty about what's missing, not a verdict downgrade. The verdict follows the predicate's normal rules; `applied-matches-spec` can still PASS in standalone mode. Pilot's policy may choose to treat standalone receipts more conservatively, but that's Pilot-side; cub-scout doesn't impose a ceiling.

### Step 2 — start the watch stream or bot

```bash

$ cub-scout watch -n prod \

--webhook https://pilot.acme/cub-scout-events \

--emit-receipt-on all \

--output-file /var/log/cub-scout-watch.jsonl \

--severity warning,critical

```

- `--webhook` is Pilot's listener (POST per event).

- `--output-file` keeps a local JSONL log (useful for replay during postmortems; see [`pilot-incident-evidence`](../pilot-incident-evidence/SKILL.md)).

- `--emit-receipt-on` attaches receipts to matching event types; use `all` with the default batch cap when first-poll bursts are possible.

- `--severity` filters scan-finding events to the actionable subset.

The watch loop is **read-only by design**. cub-scout's only cluster operations are `Get` / `List` / `Watch`; `--emit-receipt-on` adds an in-process `BuildReceipt` per matching event, which is also pure (the receipts code path is statically guarded by `TestReceiptPackageReadOnlyClient`).

### Step 3 — Pilot's per-event handler

Pilot's logic, per event:

```python

def handle_event(event: dict) -> Verdict:

# Fast path: event carries an inline receipt

if "receipt" in event:

receipt = event["receipt"]

verdict_str = receipt["predicate"]["verdict"] # PASS|WATCH|BLOCK|INCONCLUSIVE

# The receipt also carries omissions[], attribution, gitSource — Pilot can drill in

if verdict_str == "PASS":

return Verdict.PASS

elif verdict_str == "WATCH":

attach_receipt_to_ticket(receipt)

return Verdict.WATCH

elif verdict_str == "BLOCK":

escalate(event, receipt)

return Verdict.BLOCK

elif verdict_str == "INCONCLUSIVE":

return Verdict.ASK

# Slow path: no inline receipt (event type not in --emit-receipt-on, OR receipt-build failed)

return synchronous_evidence_gathering(event)

```

The slow path calls back into cub-scout synchronously:

```bash

$ cub-scout compare three-way deploy/payments-api -n prod --format json > evidence.json

$ cub-scout receipt verify deploy/payments-api -n prod --save --out event.receipt.json

```

### Step 4 — verify the receipt fingerprint (optional but recommended)

Pilot may want to verify that the inline receipt's fingerprint is intact — particularly when the event arrived via a webhook that traversed network boundaries. cub-scout's receipt validate path is exit-coded:

```bash

$ echo "$event" | jq '.receipt' > event-receipt.json

$ cub-scout receipt validate event-receipt.json

# exit 0 = fingerprint matches; 1 = mismatch (refuse to act on it); 2 = I/O

```

## Operational defaults Pilot should know

- **Receipt-build failures are non-fatal**: the event still emits, the `receipt` key is omitted (omitempty), and cub-scout logs a stderr warning. Pilot's handler must check for key presence, not null-ness.

- **Warnings are rate-limited** (Codex round-6 P2 fix in #463): the first 10 receipt-build warnings pass through verbatim; afterwards, a summary line fires every 100 suppressed warnings. A long-running watch with transient cluster-read failures won't spam stderr.

- **Unsupported event types pass through silently**: `resource.discovered` and `scan.finding` accept the flag (so `--emit-receipt-on all` works) but cub-scout skips receipt-build for them in v1. A startup warning lists them.

- **Backpressure is not yet handled** (open in `#449`): if events fire faster than the consumer drains, the watch can queue up to `--max-queued-events` and then drops oldest. ApplicationSet expansion is the canonical "events fire fast" scenario.

- **Connected vs standalone**: receipts attached to events include the `confighub-unit://` subject only when connected. Standalone-mode receipts omit it and carry an `OmissionConfigHubUnitSubject` entry — that's structural honesty about missing evidence. The verdict itself follows the predicate's normal rules (PASS is possible in standalone mode). Pilot may apply its own conservative policy; cub-scout doesn't impose a ceiling.

## Worked example: drift event triggers Pilot

A `drift.detected` event fires for `deploy/payments-api` in prod. Pilot's webhook receives:

```json

{

"type": "drift.detected",

"timestamp": "2026-05-22T11:00:00Z",

"resource": {"kind": "Deployment", "name": "payments-api", "namespace": "prod"},

"owner": {"type": "argo", "name": "payments-api"},

"severity": "warning",

"details": {"category": "DRIFT"},

"receipt": {

"_type": "https://in-toto.io/Statement/v1",

"predicateType": "https://cub-scout.dev/receipt/v1",

"predicate": {

"predicateName": "applied-matches-spec",

"verdict": "BLOCK",

"evidence": {

"attribution": {"cause": "manual-edit", "managerHint": "kubectl-edit"}

"fingerprint": "sha256:1a2b3c..."

}

```

Pilot's handler:

1. Sees `"receipt" in event` → fast path.

2. Reads `receipt.predicate.verdict: "BLOCK"`.

3. Reads `receipt.predicate.evidence.attribution: {cause: manual-edit, managerHint: kubectl-edit}` → confirms human kubectl-edited an Argo-owned resource.

4. Renders Pilot verdict: **BLOCK**.

5. Escalates: opens a ticket, attaches the receipt, pages the on-call.

Total round-trip: one webhook POST, zero synchronous cub-scout calls. The receipt is fingerprint-verifiable later if the ticket is audited.

If the same event had arrived without an inline receipt (e.g., `--emit-receipt-on` wasn't set), Pilot would call `cub-scout compare three-way deploy/payments-api -n prod --format json` to get the same evidence on the slow path.

## Open scope on the cub-scout side (issue `#449`)

This skill calls out gaps that may surface during real deployment. They live in the open scope of `#449`, not this skill:

- **More event types**: `attribution.cause-flipped` (a resource's `cause` changes from `controller-drift` → `manual-edit`); `source-truth.verdict-changed` (a `compare source-truth` verdict flips PASS → WATCH); `bindingSource.changed` (a Link now resolves to a different upstream). All v2+ scope.

- **Backpressure / batching**: high-frequency events from ApplicationSet expansion need a documented drop / batch policy. Currently `--max-queued-events` controls in-process queue depth; downstream filtering is Pilot's call.

- **Pilot-shaped emit format**: a `--format pilot` or `--shape acceptance-event` that bundles enough context per event for Pilot to verdict-render without any synchronous call-back. Receipts inline already cover the common case, but a fuller shape could include `compareThreeWay` per event.

- **`resource.discovered` / `scan.finding` receipt support**: gated on the backpressure design (would flood on initial poll burst). Open in `#449`.

Filing these is a non-goal of this skill; surfacing them during real-deployment authoring is a goal.

## Tool boundary

- - **Allowed (cub-scout):** `watch`, Compare verbs, `explain`, `trace`, `doctor`, `map`, `receipt verify / show / validate / list`. All read-only.

+ - **Allowed (cub-scout):** `watch` (not auto-approved: its `--webhook` flag sends events to any URL, so the host asks each time), Compare verbs, `explain`, `trace`, `doctor`, `map`, `receipt verify / show / validate / list`. All read-only.

- **Allowed (cluster):** `kubectl get/describe`. NOT `kubectl apply / edit / patch / delete`.

- - **Allowed (ConfigHub):** `cub * get / list`, `cub unit get`, `cub link list`. NOT `cub * create / update / delete`.

+ - **Allowed (ConfigHub):** `cub <entity> get / list` for the entities named in allowed-tools. NOT `cub * create / update / delete`.

- **Allowed (controllers):** `argocd app get`, `flux get`. NOT `argocd app sync`, `flux reconcile`.

- **Pilot's mutation path** (REVERT, QUARANTINE, escalation) is out-of-scope for this skill.

## References

- [`scout-observe`](../scout-observe/SKILL.md) — the `watch` surface from the operator side

- [`scout-verify`](../scout-verify/SKILL.md) — receipt-emit (the `--emit-receipt-on` flag wiring)

- [`scout-compare`](../scout-compare/SKILL.md) — Compare verbs Pilot falls back to on the slow path

- [`pilot-cd-gate`](../pilot-cd-gate/SKILL.md) — pre-deploy companion (query-driven)

- [`pilot-fleet-conformance`](../pilot-fleet-conformance/SKILL.md) — periodic fleet companion (query-driven)

- [`pilot-patch-and-drift`](../pilot-patch-and-drift/SKILL.md) — drill-down for BLOCK / WATCH events

- [`pilot-incident-evidence`](../pilot-incident-evidence/SKILL.md) — replay watch logs during postmortems

- Wire format: [`docs/reference/json-contracts.md`](../../docs/reference/json-contracts.md) § Receipt Contract — Watch event emission (v2.2)

- Receipts v1+v2: `#446` (v1), `#449` (this flag), `#463` (the shipping merge)

- Read-only triad: [`scout-observe`](../scout-observe/SKILL.md), [`scout-attribute`](../scout-attribute/SKILL.md), [`scout-compare`](../scout-compare/SKILL.md), [`scout-verify`](../scout-verify/SKILL.md)

## Constraints

- As of `v2.3.0` (PR #470), all four known event types build receipts: `drift.detected`, `ownership.changed`, `resource.discovered`, `scan.finding`. The per-poll cap `--emit-receipt-batch-cap N` (default 10) bounds first-poll bursts that would otherwise flood — the first N events get receipts and the rest emit with the `receipt` key omitted plus a single stderr summary line.

- The `receipt` key uses `omitempty` — consumers MUST check key presence, not null-ness. `event["receipt"]` raising `KeyError` is the correct empty signal; `if event["receipt"] is None` is the wrong pattern.

- The wire format is the **in-toto Statement v1 envelope** wrapping `https://cub-scout.dev/receipt/v1`. Pilot's receipt parser should be schema-aware to handle future envelope additions (e.g., signing layer).

- `--emit-receipt-on` requires the v2.2+ cub-scout binary (the flag shipped in `#463`); the per-poll cap requires v2.3+ (`#470`). Older binaries don't have the flag and the watch event payload won't include a `receipt` field.

- **Per-poll backpressure semantics**: the cap resets every call to `attachReceiptsIfRequested` (one watch poll = one call), so a long-running watch with quiet polls between bursts doesn't accumulate suppression state. `--emit-receipt-batch-cap 0` disables receipt-build entirely while keeping the flag explicit; large values (e.g., 1000) effectively disable the cap. See [`docs/reference/watch-events.md`](../../docs/reference/watch-events.md) for the full specification.