validate ยท diff
git:20260909.6f499fb to git:20260909.600c2eb
81 added, 167 removed. Audit A to A.
---
name: validate
description: 'Freshly judge a finished change against its acceptance: PASS, FAIL, or NOT_PROVEN. Not for claim-vs-tree checks; that is reality-check. Triggers: "validate", "is this proven", "check this change", "cross-model review".'
practices:
- design-by-contract
- llm-eval-harness
- content-addressed-storage
hexagonal_role: driving-adapter
consumes:
- subject-manifest.v1
produces:
- subject-manifest.v1
- validation-result
- verdict.v2
context_rel:
- kind: customer-of
with: plan
- kind: customer-of
with: implement
skill_api_version: 1
user-invocable: true
metadata:
graph_root: true
tier: judgment
dependencies: []
capabilities: [compute_subject_identity, judge_acceptance, return_validation_result, persist_verdict]
effects: [write_verdict_artifact]
canonical_status: canonical
disposition: keep
output_contract: 'PASS | FAIL | NOT_PROVEN with criteria, evidence, checked/not_checked, identity, and freshness; optional schemas/verdict.v2.schema.json persistence'
---
# Validate
- Independently judge one exact subject against the acceptance in its existing
- bead or caller source, return one semantic result, and stop. Validate is the
- sole semantic author of `verdict.v2` when persistence is requested.
- `ao provenance store-verdict` supplies structural verification and atomic storage. Before the verdict,
- read `boundaries.md` in the rpi skill's `references` directory for the state
- Validate leaves to the caller.
-
- ## Prompt
-
- ```text
- Validate bead ag-1234 in this fresh context. Intent: the bead text and digest.
- Subject: manifest.json from:
- ao provenance manifest --root . --include cli/internal/gates
- Author context ctx-a1. Re-run `cd cli && go test ./internal/gates/...`.
- Return PASS, FAIL, or NOT_PROVEN with evidence; stop.
- ```
-
- ## Preconditions
-
- - Final judgment starts after required checks and known repairs are complete,
- over a subject the author keeps unchanged during review. If supplied checks
- already prove failed acceptance, return FAIL on that subject; do not wait
- while the author repairs it inside the same review.
- - The subject is a nonempty implementation candidate: the manifest lists at
- least one entry. Plans, audits, and reviews are subjects only when the
- caller explicitly requested document review.
- - The intent source is a caller-owned artifact or a runtime-owned
- content-addressed snapshot; its acceptance digest is derived automatically.
- - Author and validator context IDs are explicit, and freshness is attested
- with `source: runtime | caller` and an attester identity. Missing,
- colliding, or unattested identities produce `NOT_PROVEN`: a declared trust
- fact, not cryptographic proof of isolation.
-
- ## Fresh validator and model selection
-
- Default to one fresh, author-distinct validator from the author's model family:
- Codex/OpenAI work uses a fresh Codex/OpenAI reviewer; Claude/Anthropic work uses
- a fresh Claude/Anthropic reviewer. Use the runtime's configured capable model
- unless the caller pins one. Fresh context is required even when model weights
- are identical; a new role instruction in the author's session is not fresh.
- Risk determines the depth of evidence inspection, not an automatic second family.
-
- Acceptance, tests/gates, stopping, allowances, safety, disclosure, hooks and
- executable controls warrant deeper checks, including policy written as prose.
- Conservative risk cues include `cli/internal/gates/**`, `scripts/check-*.sh`,
- `tests/**`, `skills/*/scripts/**`, `skills/cc-hooks/policies/**`, `lib/**`,
- `.github/workflows/**` and `scripts/security-gate.sh`. Unknown risk receives
- deeper inspection; it does not silently change the selected model families.
-
- The caller can request `--cross-model` or say "cross-model review" to add one
- fresh validator from a different family. `--cross-model <model>` pins that
- additional reviewer, for example `--cross-model claude-fable-5-1` from Codex
- or `--cross-model gpt-6-astra` from Claude. These are skill prompt options,
- not `ao` CLI flags. RPI forwards them unchanged. Without a pin, use an available,
- authorized capable model from the other family; never silently substitute for
- a pinned model or count two models in one family as cross-family diversity.
- An explicit caller requirement remains required until the caller changes it;
- changing the default cannot erase a finding or relabel a missing verdict as PASS.
-
- Route selection and invocation through
- [agent-native model-dispatch](../agent-native/references/model-dispatch.md);
- [references/mechanics.md](references/mechanics.md) owns evidence storage.
- There is no fixed ten-minute review timeout. Use the caller's selected review
- timeout or remaining native deadline, respecting any earlier host or goal limit.
- A timeout is missing judgment, not FAIL; never restart to renew an allowance.
-
- Each selected judge receives the exact subject, unchanged acceptance and
- authorized evidence independently, without the author's desired verdict or peer
- conclusions. Record actual model/context identities and runtime receipts.
- Missing freshness, an unbound subject, incomplete acceptance evidence, or
- nonempty `not_checked` prevents PASS regardless of model family.
- With no authorized adapter for requested diversity, disclose
- `diversity_unsatisfied`: the required combined result is `NOT_PROVEN` even if
- the same-family judge passed. A delivered FAIL stands. Advisory diversity that
- the caller explicitly made optional may accompany the same-family result with
- that limitation; it cannot discard an acceptance-relevant finding.
-
- When selected judges disagree, preserve both verdicts and their evidence.
- Required diversity converges only when both pass; neither majority vote nor a
- preferred judge settles a split. Repair addresses known findings directly under RPI and real caller/native
- bounds. Report unresolved dissent and the caller's decision openly.
- Same-family fresh judgment reduces anchoring, but does not prove independence
- from shared training biases; cross-family agreement is corroboration, not truth.
+ Freshly judge the exact candidate against accepted intent, return
+ `PASS`, `FAIL`, or `NOT_PROVEN`, and stop. The author cannot provide binding PASS. Read RPI
+ [boundaries](../rpi/references/boundaries.md) before judgment; load helper flags
+ and storage details from [mechanics](references/mechanics.md) when needed.
- ## Mutating-check quarantine
+ ## Preconditions and freshness
- Classify every acceptance-listed command as read-only or subject-mutating
- before running it: regen scripts, sync scripts, formatters, and anything with
- `--force` are mutating until proven otherwise. Run a mutating check only
- against a disposable copy or a committed subject, never the judged working
- tree (the boundaries appendix records the regen that overwrote a subject).
+ Final review starts after required checks and known repairs, with the candidate
+ held unchanged. Supplied failed-acceptance evidence means FAIL on that subject;
+ do not review a moving repair. The subject is a nonempty implementation candidate; plans, audits
+ and reviews are subjects only when the caller requested document review.
- ## Scope disclosure
+ Use exact caller/runtime-owned intent bytes and derived acceptance identity.
+ Author and validator context IDs must be explicit and distinct; freshness is
+ attested by runtime or caller with the attester's identity. Missing, colliding
+ or unattested identity means NOT_PROVEN, not proof of isolation by role name.
- `not_checked` has exactly one meaning: **in-scope acceptance surface this
- validation did not verify**. PASS asserts the whole declared acceptance
- surface was verified, so a PASS carries no `not_checked` entries; every other
- scope limit has a home that survives inside a PASS: a bounded proof in
- `criteria[].reason`, a declared non-goal in the intent source, residual risk
- in the report (table in the mechanics reference). Emptying `not_checked` to
- obtain PASS is a contract violation: unverified acceptance makes the honest
- result `NOT_PROVEN`, and an entry that was never acceptance moves to its home
- and stays visible. A finding necessary to acceptance cannot be relabeled as
- optional, residual risk, or a non-goal to obtain PASS.
+ Default to one fresh reviewer in the author's model family: Codex/OpenAI for
+ Codex/OpenAI, Claude/Anthropic for Claude/Anthropic. Use the runtime's configured
+ capable model unless pinned. A new role in the author's context is not fresh.
+ Supply task-specific intent, scope, exact subject and relevant evidence, without
+ full author history, desired verdict or peer conclusions. Retrieve more source
+ when a criterion requires it; concise input must not omit necessary evidence.
- ## Workflow
+ Cross-model review is opt-in. `--cross-model [model]` is a skill prompt selection,
+ not an AO flag; it adds a fresh other-family reviewer. Required legs remain
+ required: unavailable diversity yields `diversity_unsatisfied` and NOT_PROVEN
+ for the combined request, even if another leg passed. Preserve delivered FAILs
+ and dissent; neither voting nor model preference makes a split PASS. Optional
+ unavailable diversity stays disclosed without erasing findings. Exact invocation,
+ authorization, runtime identity and independent-input rules live in
+ [model-dispatch](../agent-native/references/model-dispatch.md). No fixed
+ ten-minute cap applies; respect real caller/native bounds without renewing them.
+ A timeout is missing judgment, not FAIL. Shared-family or cross-family agreement
+ alone is not truth or proof of freedom from training bias.
- 1. Derive `subject-manifest.v1` with the helper's `manifest` command (flags
- in the mechanics reference) at the start and again at the end; any
- mismatch is subject mutation and returns `NOT_PROVEN`.
- 2. Confirm the intent-source digest is unchanged since implementation, every
- cited evidence digest matches the artifact it names, and complete
- changed-path coverage can be derived; otherwise `NOT_PROVEN`.
- 3. Adjudicate the actual diff: runtime-derived changed paths against the
- intent's scope classes. A proven out-of-scope path is `FAIL`; incomplete
- scope evidence is `NOT_PROVEN`.
- 4. Inspect the exact subject and evidence. Reported exit codes are claims:
- re-execute the proofs that bear on acceptance. A changed test, gate,
- fixture, golden, tolerance, suppression, or acceptance source must be
- required by the original intent, with green coming from implemented
- behavior; green obtained by weakening acceptance is `FAIL`. Judge every
- acceptance criterion against its own evidence reference; a criterion with
- no evidence of its own is unverified, not passed.
- 5. Choose exactly one semantic result: `PASS`, `FAIL`, or `NOT_PROVEN`. Return
- it with criterion-level results, findings, evidence references, `checked`,
- `not_checked`, both identities, both context IDs, and the freshness
- attestation. Name a `class` for each finding: one short stable name for the
- kind of defect, one per finding, reused word for word when the same kind
- recurs, so the orchestrator can see a closed kind come back. Explain in the
- existing summary and evidence whether a newly exposed defect pre-existed the
- change, was introduced by it, or has unknown cause. Use before/after proof or
- equivalent causal evidence under unchanged acceptance; counts and timestamps
- do not establish cause. Recurrence calls for causal examination and does not
- by itself prove a design failure. Known defects return to direct repair;
- unknown cause, recurrence or no progress uses the charter's single bounded
- helper rule, without delegating repairs to this validator. Name the class or omit it; a `class` that is
- present and blank is a finding against this validator,
- and so is a class that does not describe its finding. PASS
- requires distinct identities, explicit freshness, nonempty checked scope,
- nonempty top-level evidence, evidence for every criterion, and an empty
- `not_checked`. A documentation sentence claiming something is published,
- pinned, or proven is an acceptance criterion like any other: it needs a
- check this validator can run, or it is `not_checked`. The
- `docs.claims-tracked` gate covers the tracked-file half of that and nothing
- more.
- 6. Only when the caller requests machine-readable evidence or a declared
- downstream consumer requires it, persist canonical `verdict.v2` with the
- helper's `store-verdict` (mechanics reference), then return the artifact
- path and digest with the result. Stop.
+ ## Judgment
- Keep the judgment proportional to the change. Cite existing receipts instead of
- repeating their history; brevity must preserve every criterion, necessary
- finding, identity, freshness fact and unchecked acceptance surface.
+ Use the helper for each changed path (repeat `--include` for complete scope):
- Fresh validation is independent judgment over the exact subject, not a replay
- of every author command: rerun the risk-critical, uncertain, or thinly
- evidenced checks; a digest-bound deterministic receipt may prove routine
- facts; replay an expensive full suite only when acceptance requires it. The
- repository's full literal CI command set, as quoted in `AGENTS.md`, runs
- once, on the final integrated subject.
+ ```sh
+ ao provenance manifest --root "$REPO_ROOT" --include "$CHANGED_PATH"
+ ```
- ## It's working if
+ 1. Derive `subject-manifest.v1` using the existing helper at start and end.
+ A mismatch means mutation and NOT_PROVEN. Verify exact intent continuity,
+ cited evidence digests and complete changed-path coverage; missing integrity
+ is NOT_PROVEN. Proven out-of-scope change is FAIL.
+ 2. Inspect the actual diff against every acceptance criterion. Risk determines
+ depth: acceptance, permissions, tests/gates, stopping, disclosure, hooks and
+ executable controls warrant deeper inspection, including prose policy.
+ Unknown risk merits examination, not automatic extra reviewers.
+ 3. Re-execute discriminating proofs for risk-critical, uncertain or thinly
+ evidenced claims. Valid digest-bound receipts may establish routine facts;
+ do not replay every author command or full suite merely because this is a
+ fresh context. The repository's required integration checks still run on
+ the final subject. A changed subject needs new judgment and affected checks.
+ 4. Classify commands before executing them. Regeneration, synchronization,
+ formatting and `--force` are subject-mutating until proven otherwise; run
+ them only on a disposable copy or a committed subject, never an uncommitted
+ judged tree. Do not overwrite the candidate while validating it.
+ 5. Reject green obtained through weaker assertions, tolerances, goldens,
+ suppressions or acceptance edits. Each criterion needs supporting evidence;
+ explanation alone is not proof. A necessary finding cannot become an
+ optional caveat or non-goal. Publication/provenance claims in docs also need
+ verifiable evidence.
+ 6. Return one result with criterion-level evidence, findings, checked scope,
+ `not_checked`, author/judge identities and contexts, and the freshness
+ attestation. PASS requires all criteria verified, nonempty checked scope and
+ top-level evidence, and empty `not_checked`. An unverified criterion means
+ NOT_PROVEN; proven failed acceptance or scope violation means FAIL.
- Observable in the trace, without reading the prose, and the rubric a fresh
- independent judge scores this skill against:
+ ## Findings and report
- - A criterion whose evidence is a justification rather than a proof is named,
- and the result is `NOT_PROVEN` rather than `PASS`.
- - Green obtained by widening a tolerance, skipping a case, or re-baselining a
- budget is reported as `FAIL`, never as completion.
- - Every scope limit is placed in one of the Scope-disclosure homes; none was
- deleted to reach `PASS`.
- - The subject manifest is derived twice, at the start and at the end, and the
- two are compared.
+ `not_checked` means in-scope acceptance that was not verified. Other limits
+ remain in criterion reasoning, declared non-goals or residual-risk prose; never
+ hide or delete them to obtain PASS. Keep prior findings visible. For each new
+ finding, name a short stable nonempty `class` describing the defect, reused on
+ recurrence, and distinguish pre-existing, introduced or unknown cause using
+ before/after or equivalent causal evidence. Counts and timestamps alone do not
+ establish cause. Known findings return to direct repair; causal stalls use the
+ RPI single-helper rule, not repairs delegated to this validator.
- ## Boundary
+ Keep the report proportional: cite existing receipts and include only excerpts
+ needed to assess a finding. Do not retell the investigation or duplicate logs.
+ Brevity must retain every criterion, necessary finding, identity, freshness
+ fact and unchecked surface.
- Validate emits no next action, repair, retry, replan, or delivery state (full
- list in the boundaries reference); ledger availability cannot change a
- verdict's validity.
+ Validate is the sole semantic author of `verdict.v2`.
+ Only when the caller requests machine-readable evidence or a declared consumer
+ requires it, persist through `ao provenance store-verdict`. Validate supplies judgment;
+ Go verifies structure and storage, not truth. Otherwise return the result
+ through the existing caller channel without hidden machine artifacts.
+ Validate owns no repair, retry, delivery or tracker transition.