git:20260507.8dfdcb4 to git:20260507.9f5be95

---

name: eval-harness

description: Evaluation harness for testing agent and skill quality through structured benchmarks, regression tests, and quality scoring.

allowed-tools: Read, Write, Edit, Bash, Grep, Glob

graph:

domains: [domain:software-engineering]

skillAreas: [skill-area:agentic-loops, skill-area:orchestration-loop]

workflows: [workflow:feature-development]

topics: [topic:developer-experience]

roles: [role:tech-lead, role:backend-engineer]

- Define test cases with known-correct outputs

- Run agent against each test case

- Score: accuracy, completeness, relevance

- Compare against baseline performance

- Track performance over time

### 2. Skill Quality Testing

- Verify skill instructions produce expected outcomes

- Test edge cases and boundary conditions

- Measure consistency across multiple runs

- Check for harmful or incorrect outputs

- Validate against ground truth

### 3. Regression Suite

- Collection of previously-passing test cases

- Run after any agent/skill modification

- Flag regressions with before/after comparison

- Maintain pass rate threshold (>= 95%)

### 4. Process Verification

- End-to-end process execution with known inputs

- Verify each phase produces expected outputs

- Check task ordering and dependency satisfaction

- Measure total execution time

## Quality Scoring

### Accuracy Score (0-100)

- Correctness of output vs expected

- Partial credit for partially correct outputs

- Penalty for hallucinated or fabricated content

### Completeness Score (0-100)

- Coverage of required output elements

- Missing sections flagged and scored

- Bonus for useful additional context

### Consistency Score (0-100)

- Run same input 3 times

- Compare outputs for semantic similarity

- Flag inconsistencies

### Composite Score

- (accuracy * 0.4 + completeness * 0.3 + consistency * 0.3)

- Threshold: 80 to pass

## When to Use

- After creating new agents or skills

- After modifying existing agents or skills

- Periodic quality audits

- Before promoting skills to production

## Agents Used

- Used by process-level evaluation orchestrators

- No specific agent dependency (evaluates other agents)