DESIGN.md@docs · diff
git:20260623.afba4e4 to git:20260703.8abb4f9
10 added, 5 removed. Audit A to A.
# /supergoal — design rationale
Why each part is built the way it is. Sources: the local autopilot/ultrawork/ultraqa/ultragoal
skills (reusable mechanisms) and a fact-checked web research brief (`docs/research-brief.md`,
18 agents, adversarially verified).
- Status note (2026-06-24): the active executable contract is `SKILL.md`, `reference/role-loop.md`,
- `templates/qa-gate.sh`, `templates/qa-only-gate.sh`, `templates/harness-eval-gate.mjs`,
- `templates/skill-frontmatter-gate.mjs`, and `tests/run-all.sh`. References below to
- `delivery-gate.sh` and `human-feedback-gate.mjs` are historical validation records from earlier
- revisions, not live gates in the current skill.
+ Status note (2026-07-03): the active executable contract is `SKILL.md`, `reference/role-loop.md`,
+ `reference/delivery-gate.md`, `templates/commit-gate.sh`, `templates/qa-gate.sh`,
+ `templates/qa-only-gate.sh`, `templates/harness-eval-gate.mjs`,
+ `templates/skill-frontmatter-gate.mjs`, and `tests/run-all.sh`. The live loop is now Build ->
+ Forced Verify as the mandatory core; Critic/Fixer is an opt-in escalation for under-specified or
+ latent-correctness work. The current evidence frontier is the production-adoption plan in
+ `docs/changelog/2026-07/02-production-adoption/plan.md`, not another automatic rewrite of the
+ historical design below. References below to removed gates such as `delivery-gate.sh` and
+ `human-feedback-gate.mjs` are historical validation records from earlier revisions, not live gates in
+ the current skill.
## Core design (gated lanes over a shared vault)
The spine is **forward-only gated lanes**: a single shared **vault** as the only cross-phase state,
an **untrusted `claims.md` re-verified by an adversarial Verify** that re-runs every claim, **role
separation by read-scope**, and literal executable gates for active lanes that are never edited to pass.
**Self-contained by design:** no required service, TUI, or external WORKFLOW files.
`/supergoal` runs **in-session with subagents**; the Board is optional observability only. (Reuse-map
decision: keep the gate ideas, skip the heavyweight orchestration machine.)
**Validate front-lane:** a dedicated demand-validation step gates GREENFIELD before any Build ticket
opens — cheap evidence first, code second — so the skill never builds something nobody wanted.
## Reused from local skills
- **autopilot** → phase pipeline + artifact-skip gates + bounded-retry circuit breaker + multi-expert
parallel validation before delivery.
- **ultraqa** → diagnose→fix split (Opus diagnoses, Sonnet fixes) + same-error-3× circuit breaker.
- **ultrawork** → parallel-wave dispatch + tiered model routing (Haiku/Sonnet/Opus) + background >30s.
- **Frontmatter/house style** → portable `name` + trigger-rich `description`;
thin `SKILL.md` spine + `reference/*.md` loaded on demand (progressive disclosure).
- **taste-skill v2** (leonxlnx/taste-skill, `design-taste-frontend`) → the design authority for UI/UX
jobs. Vendored **verbatim** as `reference/taste-skill-v2.md` (only file v2 needs; companions like
v1/output/imagegen are not required), under a provenance banner so refresh is a body-swap, not a
merge. A thin `reference/ui-ux.md` overlay loads it on demand at Plan/Build/QA — keeping the
always-on spine lean while UI work gets full anti-slop guidance. Why vendor vs. submodule: pulls one
file not the whole repo, and a pinned commit makes drift auditable.
## Grounded by research (each decision → evidence)
| Decision | Evidence |
|---|---|
| Topology classifier: fan-out only for wide-shallow; single-driver for deep-narrow (DEBUG/LEGACY) | Cognition + Anthropic mid-2025 convergence; LangChain task-topology |
| Orchestrator ingests subagent **summaries**, never raw transcripts | Anthropic multi-agent system; flowhunt.io; LangChain |
| Locked role-scoped subagents, pinned model+prompt (critic ≠ coder context) | arxiv 2507.19902 AgentMesh; arxiv 2506.17208 |
| Read-only Plan Mode + Human Feedback approval before writes | antstack.com; developersdigest.tech |
| **Two-layer gate**: hard tests (deterministic) + soft LLM rubric; rubric never overrides red tests | arxiv 2506.17208 (Anthropic uses both) |
| Committee of diverse reviewers (correctness/security/maintainability) | arxiv 2511.16708 Codex-Verify; arxiv 2506.17208 |
| Gate on the project's own suite in a clean sandbox, never benchmarks/self-report | SWE-bench contamination — arxiv 2509.16941; morphllm.com SWE-bench Pro |
| Validate the suite itself (failing-before requirement / mutation check) | 59.4% of SWE-bench hard tasks have flawed tests — arxiv 2509.16941 |
| DEBUG = reproduce-first (red), root-cause, passing-after in clean sandbox | Anthropic verification practice; arxiv 2509.16941 |
| Honest product stance: supervised gated loop, not autonomous engineer | Devin ~14-15% unassisted on complex tasks — Answer.AI; cognition.ai |
| Cost: token spend ≈80% of multi-agent variance → gate fan-out, trim boilerplate | Anthropic multi-agent system |
**Excluded (refuted in verification):** the "4-6× single-agent token overhead / ~40-50 lines per
child" figures — single-RFC, mislabeled empirical, misattributed. Only Anthropic's ~80%-variance
sub-fact is retained.
## Verification of this skill
Historical validation: earlier revisions tested `templates/delivery-gate.sh` across 5 scenarios
(missing artifacts -> FAIL; complete+GREEN+passing -> PASS; RED verdict -> FAIL; NO-GO -> FAIL; failing
suite -> FAIL), and later tested `templates/human-feedback-gate.mjs` for pre-implementation approval.
Those gates were removed when the current baseline-first contract replaced them with narrower active
lane gates: QA, QA-ONLY, HARNESS-EVAL, LEARN-DOMAIN, DB read-only, and skill frontmatter.
### Live end-to-end validation (2026-05-29)
Ran a full GREENFIELD pipeline on a real production-grade objective ("build a production URL
shortener service and ship it"): Validate(GO) → Plan(frozen) → Human Feedback → Build → Verify → Committee → QA →
Deliver. The harness paid off exactly where designed — the **builder≠verifier separation caught real
bugs the builder's own 43 green tests missed**:
- Verify cycle 1 → RED: 2 genuine SSRF bypasses (`[::ffff:127.0.0.1]` IPv6-mapped loopback,
`localhost.` trailing-dot) → rewind to Build → fixed → Verify cycle 2 GREEN.
- Committee (architect/security/code-review all APPROVE) surfaced 1 MEDIUM (malformed
percent-encoding → unauth 500) → fixed → Verify cycle 3 GREEN.
- QA 11/11 black-box, then `delivery-gate.sh` exit 0 (51/51 tests in clean state), then commit.
A single unrouted agent would almost certainly have shipped the SSRF holes on a "43 tests pass".
### Live DEBUG-mode validation (2026-05-29)
A realistic lost-update race was injected into the shortener's `incrementHit` (read hoisted out of
the mutex) as a debugging-drill fixture; the bug was left UNCOMMITTED (an earlier attempt to disguise
it as a "perf" commit was correctly blocked by the content-integrity classifier — the deceptive commit
was reverted, not worked around). The DEBUG harness, given only the symptom ("popular links undercount
hits under concurrency"), ran single-driver + read-only Plan Mode: reproduced (200 concurrent →
1/200), root-caused with competing hypotheses + a git-stash discriminating probe, **stopped at the
Human Feedback**, then fixed + adversarially verified (52/52 + 10 concurrency trials, 0 lost). Committed
as `0ce82a1`.
### Live LEGACY-mode validation (2026-05-29)
Added optional link expiry (TTL) to the now-real codebase. Explore mapped the change surface
(file:line) read-only; a frozen surgical plan was approved at Human Feedback (expired → 410 Gone); Build
implemented it; adversarial Verify confirmed **no regression** (redirect still counts hits after the
get→increment reorder, the DEBUG concurrency guard still 200/200, unknown still 404, legacy records
with no `expiresAt` never expire) at 68/68; security + code-review committee APPROVE (LOW only); gate
green. Committed as `b2c897f`. Demonstrated the LEGACY-specific value: backward-compat + surgical
blast radius + regression-safety on top of existing tests.
All three modes are live-validated. Adversarial verification caught a real defect in 2 of 3 runs
(SSRF + concurrency race); the LEGACY run shipped clean because Explore-first + a frozen surgical plan
kept the change correct on the first build.
> Note: the commit SHAs above (`c3d74f6`, `0ce82a1`, `b2c897f`) refer to the ephemeral
> `/tmp/jdi-live/url-shortener` working copy used during validation, not this repository's history.
> The validated source + full audit trail are vendored under `examples/url-shortener/`.