testbench-gen git:20261102.b666e3dA

Immutable. This exact content never changes and is served at /api/v1/blob/defd50ab13fa6274.

---
name: testbench-gen
description: "Generate a PROFESSIONAL, high-coverage testbench for a design directly from its Phase-1 L-docs — the Phase-2 'Simulation' step (canonical flow Step 4). Use whenever a design needs functional verification: derive the interface (L1/L9), clock/reset (L8/L9), a reference model, a scoreboard, functional coverage, and assertions, then run cocotb. Program-first: the deterministic generator `professional_tb_gen.py` (MCP `eda_professional_tb`) does the derivation; this skill is the judgment layer for filling a non-closed-form reference model and reviewing coverage. Triggers on: 'write a testbench', 'generate testbench', 'verify this design', 'functional verification', 'coverage', '幫我寫 testbench', '產生 testbench', after Phase-2 RTL is authored and before Phase-3."
---

# testbench-gen — deterministic professional testbench generation

The canonical flow's **Step 4 (Simulation)** derives a professional, high-coverage
testbench from the Phase-1 layers — never a hand-waved skeleton. The load-bearing
work is a PROGRAM: `programs/professional_tb_gen.py` (exposed as MCP tool
`eda_professional_tb`). This skill is the judgment layer around it.

## What the program does (program-first)

`professional_tb_gen.py <project>` reads the project's `phase1/generated_docs/L*.json`
and emits, under `phase2/stage1/sim_professional/<top>/`:

- `tb_<top>.py` — a **cocotb** testbench: clock/reset from L8/L9, a reference
  model, a SCOREBOARD, functional coverage (cocotb-coverage covergroups), and a
  directed-corner + constrained-random stimulus loop.
- `<top>_coverage_model.json` (an **L28** coverage model), `<top>_assertions.sva`
  (an **L29** assertion set), a cocotb `Makefile`, and `verification_plan.json`.

It **combines open-source** frameworks rather than re-inventing them: **cocotb**
(TB base) + **cocotb-coverage** (functional coverage + constrained-random) +
**Verilator/Icarus** (sim) + an **SVA bind** for Verilator/SymbiYosys. Add
`cocotb-coverage` to the runtime (see mcp-eda/INSTALL_GUIDE.md).

## The reference-model strategy — 3 tiers (honest, never a vacuous pass)

1. **Closed-form** (arithmetic primitives `c = a OP b`): reuses
   `arith_oracle_tb_gen.compute_golden` — the single source of truth.
2. **Bounded-latency streaming scoreboard** (serial datapaths — one parallel
   operand + a 1-bit serial operand + a 1-bit serial result, e.g. a bit-serial
   multiplier): the scoreboard AUTO-DERIVES the output `(latency, bit_order)`
   from a calibration vector, then checks every vector against the closed-form
   reference. **This closes the serial-datapath DEFER** that `arith_oracle_tb_gen`
   punts on entirely (proven: spm bit-serial multiplier, 208/208 vs
   `(x*y) mod 2^N`, `order=lsb latency=1` auto-derived).
3. **Reference hook** (classes with no closed form — CPU/SoC/DSP): the generated
   TB emits a clearly-marked `reference_model` hook that **`TestSkip`s until
   filled** — NEVER a silent vacuous pass. The judgment layer (this skill /
   spec-to-refmodel) fills it from L10 vectors or a spec-derived model, with the
   program-vs-AI cross-check (dual-track convergence).

## When YOU (the judgment layer) act

- **Fill a Tier-3 reference model**: read the L-docs (L2 function, L3 protocol,
  L10 vectors) and author the Python reference the hook needs — then re-run so
  the scoreboard is real. This is the only place an LLM is needed; the interface,
  clock/reset, coverage bins, and (for Tier-1/2) the reference are all derived by
  the program.

  **WRITE IT TO `expert_reference_tb.py`, NOT TO `tb_<top>.py`.**

  ```
  phase2/stage1/sim_professional/<top>/expert_reference_tb.py
  ```

  `tb_<top>.py` is REGENERATED by `professional_tb_gen` on every Step-4 run, so
  an oracle written there is deleted by the very re-run the gate asks for —
  `cpu_functional_oracle_waiver_check` says "fill … with the testbench-gen
  expert fallback and re-run Step 4", and that loop has no fixed point if you
  edit the generated file. The sibling `expert_reference_tb.py` is deliberately
  stored apart so regeneration cannot erase your work;
  `professional_tb_gen._load_expert_reference_tb` reads it and promotes the run
  to `dut_kind=expert_reference`, which is what makes the Phase-2 runner
  actually launch cocotb for an otherwise-generic class.

  For the loader to ACCEPT it (it rejects placeholder shapes before a simulator
  is launched), the file must:

  | requirement | why |
  |---|---|
  | a `@cocotb.test()` coroutine that OBSERVES `dut` and carries a real `assert` | a testbench that never reads the DUT is not an oracle |
  | print `PROFESSIONAL_TB PASS` from a passing check | the marker the Phase-2 runner greps for |
  | contain no `TestSkip` | a skipped hook is the unfilled state |

  The JUnit is still the authority: the run is credited only on
  `tests>0, passed>0, failures==0, errors==0`. The marker never credits a case
  on its own.

  In a benchmark run, an AI backup task whose evidence window reaches Step 4
  names this path in `reference_model_handoff`. Fill it before `--resume` so the
  first re-gate measures it. A model filled or replaced later is measured by the
  next `--resume`, which re-enters the runner on the same RTL bytes and issues a
  fresh review. A model the last run already measured is not re-run.

  *Measured (2026-09-02, a subservient × gf180mcuD acceptance run): the whole
  convention was named in exactly two files — `professional_tb_gen.py` and its
  own test. No gate message, no skill and no doc mentioned it, so an agent
  following the gate's own instruction wrote a full 7-oracle testbench into
  `tb_<top>.py` and watched the next Step-4 run delete it.*
- **Review coverage**: after a run, read `coverage_<top>.xml` + the L28 model;
  if functional bins are unhit, add directed vectors or tighten the
  constrained-random constraints. Loop until the closure policy (functional bins
  100% + code coverage targets) is met.
- **Formalise assertions**: the L29 SVA carries reset→known-state + protocol
  handshake properties deterministically; L16 must/shall prose is emitted as
  `TODO(spec-to-assertion)` stubs for you to formalise (never fabricated as a
  passing SVA).

### Skill: Software-visible field profiles for reference models

**Pattern**: An instruction extension or interface selection is treated as proof of every software-readable or writable field, so a reference model confuses internal saved state with the selected software interface.

**When to apply**: Authoring a Tier-3 model around an implementation that exposes a documented subset of architectural or interface state.

**What to do**: First record the declared scope, selected IP revision and pinned public capability sources, then distinguish software read masks, write permissions and side effects from internal state. Compare only source-supported fields while preserving transaction widths, timing, full byte commits and unknown-value checks. Check an internal saved state through its specified observable effect when software readback is not promised. An unsupported or ambiguous requirement remains `NOT_MEASURED` and cannot be silently removed from required acceptance. If a newly reviewed profile narrows an earlier oracle, preserve the old oracle and counterexample and label the new scope explicitly; never choose masks or goldens from DUT traces. Retain positive, fault-specific negative and byte-identical reverse controls.

**Worked pattern** (anonymized): An interrupt implementation may retain a saved enable internally for return behavior while exposing only the current enable through software reads. Test the guaranteed read field and the specified return effect independently; do not invent read or write access to the saved field. A second eventual interrupt alone does not prove that pending state survived masking: use a discriminating countercontrol or keep that property unmeasured.

**Why this is GENERAL**: The same distinction applies to processor status fields, peripheral register views and protocol capability subsets. Instruction or interface membership, internal state and software access are separate claims; actual native/JUnit evidence remains scoped to the tested fields and effects.

_Captured by benchmark-enhancement-capture 2026-10-15._

### Skill: Mode-conditioned meaningfulness in reference oracles

**Pattern**: A software-visible field is explicitly meaningful only in selected modes or states, but a Tier-3 reference oracle asserts one exact value unconditionally and rejects another source-permitted implementation.

**When to apply**: Authoring or reviewing a full-stack expert reference when `verification_plan.json` carries `mode_conditioned_field_profiles`, or when INPUT describes a register or protocol field as not meaningful, undefined, or don't-care under a named condition.

**What to do**: Build a field-by-mode profile from the cited INPUT statement. When its condition holds, do not assert an exact value for that field. Continue to check every source-defined peer field, transaction timing, known-value requirement and observable effect. In each mode where the field is meaningful, retain its complete source-supported behavior. Keep two legal-value positive arms for the non-meaningful mode, a meaningful-mode negative control, and a reverse mutation of a neighboring defined field. Derive neither the condition nor an expected value from DUT traces.

**Worked pattern** (anonymized): A status bit reports lost output in manual overwrite mode but is explicitly not meaningful in automatic stall mode. Automatic readback zero and one are both legal for that bit. Manual loss behavior, output-valid, idle and stall remain checked. Removing all checks for the register is a vacuous repair and must fail the peer-field reverse control.

**Why this is GENERAL**: Mode-dependent status and capability fields occur in accelerators, peripherals, CPUs and mixed-mode controllers. The producer emits source-cited profiles without a design name, address, bit number or inferred trace value; the expert review applies them to arbitrary Tier-3 oracle code.

_Captured by benchmark-enhancement-capture 2026-10-21._

### Skill: State-local control independence

**When to apply:** a protocol or FSM uses different controls in different
phases. Derive a dependency table from the admitted input: which controls can
change the next state, stored data and outputs in each phase?

**What to do:** exercise both values of relevant controls and verify that
irrelevant controls do not disturb progress. If unknown/high-impedance controls
are legal in that phase, also drive X/Z on only the irrelevant controls while
keeping relevant controls known. Include an unchanged binary control and a
relevant-unknown negative/default control. Do not invent an initial state,
reset, undefined-input output value or universal four-state requirement.
For a complete explicit Moore table, state-local Boolean support reduction is
enforced by `programs/moore_fsm_table_emit.py`; an incomplete table must still
refuse emission. Apply the behavioral dependency checks to AI-authored RTL too.

**Why not Bucket A:** the complete-table optimization is already a program.
Determining legal unknown-input domains and dependencies in arbitrary prose
requires source-bound judgment. A binary test or a port-reference coverage
number alone does not establish phase independence.

## Optional reset-retention byte stimulus

When a retention test needs a designer-chosen memory stimulus, declare
`verification_stimuli.reset_retention.memory_preload` in the project's
`plugin_output/declaration.json` through `spec_declaration_emit --from-json`
or `--set`. This verification choice is separate from the SRAM timing contract.
It is a non-empty list of `{"address": 0, "byte": 165}` entries: exactly those
two keys, unique addresses within the selected memory depth, and integer byte
values 0..255. Boolean, string and floating-point values are refused.

`testbench_gen.emit_unit_tbs` applies the bytes after initializing every memory
location and before observing either reset arm. It preserves every rising-edge
write, the pending first-edge check, all-byte comparisons and the 64-cycle
activity budget. Absent preload keeps the stock seed; malformed preload cannot
credit actual-byte retention. A chosen stimulus alone is never execution
evidence: absent non-idempotent pending activity remains NOT_MEASURED.

## Declared serial arithmetic scenarios

The existing unit producer can generate operand-boundary, consecutive-transaction
and synchronous-reset scenarios from the case's own L10 scope. It requires the
selected serial function, bit order and settled observation offset, plus
`transaction_rearm`, `rearm_reset_cycles` and
`parallel_hold=through_final_product_bit`. Supported rearm is `clocked_reset`,
or `continuous` when the settled offset is zero. Unmodeled input roles, clock
domains, reset styles and ambiguous scopes remain unmeasured.

The scenario oracle uses independent modulo goldens at one declared window; it
does not search DUT alignment to choose an expected answer. It exercises all
selected corner pairs, the stated transaction population, early/middle/late
interruptions with single/multiple reset holds, and between-edge reset pulses.
A row listing multiple supported families executes each family in the same
native case, with separate actual populations and an aggregate comparison total.
Generated scenario credit requires the current native comparison population
and unchanged input, DUT, declaration, TB and transcript identities. Generic
generated arithmetic remains insufficient for scenario credit. Authored and
delivered testbenches retain their original priority.

`latency_cycles_origin=settled_edge_offset` is the normal CR-5 convention.
Explicit legacy `pre_nba_sample_offset` is normalized once. Native manifests
carry their origin; a bare measured integer is not origin evidence. Neither
convention relaxes an incorrect declared window.

## Capture

Any recovery here (a new reference-model pattern, a new coverage-derivation rule,
a streaming-alignment case) is captured per `benchmark-enhancement-capture`.
Route full-stack professional-testbench recovery to `professional_tb_gen.py`.
Route an exact-name L10 `.v` / `.sv` unit oracle from
`testbench_gen.run_unit_tbs` through the canonical
`phase2.l10_unit_tb.testbench_gen.run_unit_tbs` entry: Bucket A belongs to
`testbench_gen.py` with `_l10_execution.py` as its receipt owner, and Bucket B
belongs in this skill. Do not borrow the full-stack route for a unit-TB record.

For L10 unit-oracle recovery, keep the authored TB bytes and bind the complete
source closure needed to reproduce its expected result. A wrong expected value
or wrong stimulus must still compile and then fail the native comparison; the
restored original bytes must pass through the normal producer and write a fresh
execution receipt. A broad reference-suite PASS cannot credit a named vector it
did not execute. If the gap is in a forked simulator (Verilator/Icarus/cocotb),
also emit a **Bucket-T** forked-tool item.

## Gaps to grow into (from the 2026-07-11 verification research)

- Fill the currently-empty **L22 verification plan** fields (coverage_goals /
  formal_properties / regression_matrix) and wire them to L28/L29.
- Add **L30 stimulus/constraints** (constrained-random weights, sequences).
- Widen code coverage to FSM-transition / expression / condition (Verilator
  `--coverage-user`) and gate sign-off on it.
- pyuvm structural env + riscv-dv for CPU-class constrained-random.

## Compliance gate (mandatory)

After producing your output, save it to a file and run:

```bash
python3 plugins/vibe-ic/_shared/skill_compliance_check.py \
    --requirements plugins/vibe-ic/skills/testbench-gen/compliance.yaml \
    <your_output_file>
```

Exit 0 = PASS, exit 1 = FAIL with specific missing elements listed.
`compliance.yaml` in the corresponding skill directory enumerates
every required element of your output: section headers, metadata fields,
handoff lines, tool invocations.

**Your task is not complete until the audit returns PASS.** Missing
elements are the single largest source of skill-execution non-determinism
across different agents.

### Skill: Legal delayed acknowledgement withdrawal with public observation

**Pattern**: Nominal service can complete while leaving a source-permitted request-low acknowledgement tail untested.

**When to apply**: Native coverage identifies that branch and the original interface and reset prose permit bounded delayed withdrawal with stable payload.

**What to do**: Read the complete handshake and reset obligations before choosing a finite transaction schedule; do not transfer a reset-only timing requirement to every transaction. Admit the normal design class and expert-reference contract first. Give one owner to the public service coroutine, use a separate public-pin monitor to check request remains low and acknowledgement, validity and payload remain stable throughout the selected tail, then withdraw acknowledgement and restore the existing service even on failure. Check subsequent progress against an independently specified result from the input, never an expected value inferred from the DUT. Keep the original vector and control populations, their order and goldens intact; append a named case in the persistent expert source with its complete source closure. A wrong independent expected result must compile and reach a real native comparison failure; exact restoration must pass. A zero-tail control may isolate coverage causality but does not qualify delayed-tail behavior. Hand the complete source back to the normal professional generator and native/JUnit consumers for fresh full-population measurement.

**Why not Bucket A**: The interface prose permits several finite service schedules and does not uniquely select an observable progress witness. Those choices require source-bound engineering judgment; existing programs retain source, population, execution and coverage validation.

**Worked pattern** (anonymized): A delayed-tail case passed with a separate public monitor and an input-derived functional golden. Changing one expected result caused a native comparison failure and restoring the exact source passed. An identified native branch counted three, zero and three in delayed-tail, zero-tail and restored arms without changing the coverage denominator. A driver delivery count alone did not prove internal one-time consumption.

**Why this is GENERAL**: Applies to request/acknowledgement interfaces with source-supported payload stability, legal bounded timing and independently observable progress. It establishes the measured public behavior and selected branch, not complete internal consumption, clock-domain signoff or whole-design qualification.

_Captured by benchmark-enhancement-capture 2026-10-21._

# Bucket B — skill sections to APPEND to skills/testbench-gen/SKILL.md

### Skill: Derive event phases before an executable timing challenge

**Pattern**: A timestamped table is silently treated as either pre-edge input values or post-edge drives, so an executable challenge shifts acceptance and completion by one cycle while appearing precise.

**When to apply**: Reviewing a generated reference model, waveform replay or proposed executable challenge whose visible prose and timing rows must jointly define synchronous behavior.

**What to do**: Keep Program First: run `professional_tb_gen.py` through the normal producer and freeze its exact output and raw evidence. An independent IC expert then derives from the whole visible contract the input drive phase, capture edge, settled sample point, output-valid interval, transaction overlap, reset behavior and any condition-specific don't-care. Cite the source for each decision and distinguish requirements from assumptions. Where a timestamp admits pre-edge values and post-edge drives, explain both schedules and their observable consequences; choose the best source-bound interpretation using all visible prose, diagrams and tables before declaring ambiguity. Neither a table timestamp alone nor simulator scheduling, a DUT trace, prior score or hidden oracle supplies normative authority. Encode the chosen phase explicitly in the challenge, avoiding a drive/sample race and observing settled outputs. A disagreement still needs a prompt-derived executable proof that fails the frozen Program output; repair must pass the same unchanged challenge, normal gates and a fresh independent review. Do not apply a universal one-cycle correction or grant automatic PASS. If independent adjudication leaves two interpretations equally supported, preserve both and report the affected timing claim `NOT_MEASURED` with the exact missing convention; keep separately determined checks in scope. This review guidance prevents an error and does not itself prove a functional recovery.

**Worked pattern** (anonymized): A neutral contract has a 10 ns clock, a request row at a rising timestamp of 40 ns, and completion two full periods after request acceptance. If the row describes the value present before the edge, acceptance is at 40 ns and completion at 60 ns. If it describes a drive after the edge, acceptance is at 50 ns and completion at 70 ns. Visible prose saying the request is stable before the listed edge selects the first schedule; a table without that statement requires the rest of the contract to adjudicate. Define the settled observation point and valid interval in either schedule instead of shifting every expectation by one cycle.

**Why this is GENERAL**: Timestamped synchronous specifications occur in controllers, pipelines and streaming interfaces. The reasoning concerns event semantics and source authority, not any design identifier or stored solution. Existing generators and execution consumers retain their proof responsibilities.

_Captured by benchmark-enhancement-capture 2026-11-02._

### Skill: Complete the declared expert reference fallback

**Pattern and application**: A skipped professional testbench can expose an unfilled expert reference hook without proving an RTL defect. When `professional_tb_gen.py` reports `NOT_MEASURED` with a `testbench-gen` fallback, inspect the actual skip cause and producer evidence. The IC expert resolves evaluation semantics from the visible contract instead of asking the human to supply the model.

**Model derivation**: After the producer's typed handoff, fill only its declared `expert_reference_tb.py` with an independent input-derived model. Derive operand and result widths, signedness, overflow and flag ordering, reset behavior, deassertion and settled observation phases from that contract. For a wrapping result, determine whether the flag observes the width-limited value; for selection, establish the specified bit significance. Check source-promised known values and latch behavior after deassertion: an X-only check can miss a wrong known value. Expected results must not come from DUT behavior or a hidden oracle. Preserve the prior candidate, receipts and reviews.

**Worked pattern and scope**: An absent hook produced one genuine JUnit skip and no passes or failures. Filling the declared hook with an independent model allowed the same frozen RTL to execute; substantive wrong-behavior controls failed without skipping and exact restoration passed. These are component proofs. Rerun the normal producer and require actual non-skipped JUnit comparisons, its success markers, discriminating wrong-behavior controls and a fresh independent review. Retain required gates and do not perturb RTL to force entry. Model derivation from arbitrary visible prose requires engineering judgment; existing programs own handoff and execution validation. Component evidence cannot credit canonical handoff, regating, whole-flow acceptance or a benchmark score. _Captured by benchmark-enhancement-capture 2026-11-02._