scoring-checks · git:20260319.c97e02c · 2026-03-19 · sha256 642f8b0d3f6dfea4

scoring-checks git:20260319.c97e02cA

Immutable. This exact content never changes and is served at /api/v1/blob/642f8b0d3f6dfea4.

No tool to install first
claude plugin marketplace add https://markdownregistry.com/mp/a/art_fqnnd7l7rhwuhjt2/642f8b0d3f6dfea4383049a7404b15f5f51896179b257207f50f42d6f2926a04.json

Then /plugin install scoring-checks@markdownregistry-fqnnd7l7rhwuhjt2 in Claude Code. It installs this exact version, the one graded A here, pinned to the commit it was graded at, and leaves it there.

---
name: scoring-checks
description: Adds a new deterministic scoring check to src/scoring/checks/ implementing the Check interface from src/scoring/index.ts. Point values from src/scoring/constants.ts. Checks receive dir:string and return Check[]. Use when user says 'add scoring check', 'new check', 'score X', or modifies src/scoring/. Do NOT use for LLM-based scoring, real-time validation, or checks requiring external API calls.
---
# Scoring Checks

## Critical

1. **Checks are deterministic and synchronous.** No LLM calls, no async operations, no external APIs. Use only filesystem reads and sync analysis.
2. **Every check must implement the `Check` interface** from `src/scoring/index.ts`:
   ```typescript
   export interface Check {
     id: string;           // kebab-case, e.g., "claude-md-exists"
     label: string;        // Human-readable, e.g., "CLAUDE.md exists"
     points: number;       // From src/scoring/constants.ts
     passed: boolean;      // true/false
     details?: string;     // Optional explanation
   }
   ```
3. **Points must be defined in `src/scoring/constants.ts` BEFORE the check.** Do not hardcode point values in the check file.
4. **Each check file must export a function with signature:** `export function checkName(dir: string): Check[]` that returns an array of Check objects.
5. **Import and register the check in `src/scoring/index.ts`** in the `runChecks()` function.

## Instructions

1. **Define point value in `src/scoring/constants.ts`**
   - Add constant: `export const CHECK_NAME_POINTS = <number>;`
   - Example: `export const CLAUDE_MD_EXISTS_POINTS = 15;`
   - Verify the constant is exported and value aligns with project scoring scale (typically 5–50 points per check)

2. **Create the check file at `src/scoring/checks/<check-name>.ts`**
   - File name must be kebab-case (e.g., `claude-md-exists.ts`)
   - Verify the file path follows existing pattern: see `src/scoring/checks/existence.ts`, `quality.ts`, etc.

3. **Implement the check function with the Check interface**
   ```typescript
   import { Check } from '../index';
   import { CHECK_NAME_POINTS } from '../constants';
   import { existsSync } from 'fs';

   export function checkName(dir: string): Check[] {
     const filePath = `${dir}/path/to/check`;
     const passed = existsSync(filePath);
     return [{
       id: 'check-id',
       label: 'Human label',
       points: CHECK_NAME_POINTS,
       passed,
       details: passed ? 'Found.' : 'Not found.'
     }];
   }
   ```
   - Use `existsSync`, `readFileSync`, `statSync` from `fs` module (sync only)
   - Use `glob.sync()` for multi-file patterns
   - Return an array (even single checks must be wrapped in `[]`)
   - Verify the check handles missing files gracefully (returns `passed: false`, not throws)

4. **Import and call the check in `src/scoring/index.ts`**
   - Add import: `import { checkName } from './checks/<check-name>';`
   - Add call in `runChecks(dir: string)` function:
     ```typescript
     const checks: Check[] = [];
     checks.push(...checkName(dir));
     ```
   - Verify the function is called before the final return statement

5. **Write tests in `src/scoring/__tests__/<check-name>.test.ts`**
   - Use Vitest and `memfs` for isolated filesystem
   - Example:
     ```typescript
     import { describe, it, expect, beforeEach } from 'vitest';
     import { vol } from 'memfs';
     import { checkName } from '../checks/<check-name>';

     describe('checkName', () => {
       beforeEach(() => vol.reset());
       it('passes when file exists', () => {
         vol.fromJSON({ '/project/file.md': 'content' });
         const result = checkName('/project');
         expect(result[0].passed).toBe(true);
       });
       it('fails when file missing', () => {
         const result = checkName('/project');
         expect(result[0].passed).toBe(false);
       });
     });
     ```
   - Verify both pass and fail cases are tested

6. **Run scoring tests to validate**
   - Execute: `npm run test -- src/scoring`
   - Verify all new tests pass and the check integrates without errors
   - Verify the check appears in `caliber score` output

## Examples

**User says:** "Add a check that verifies CLAUDE.md exists and has content"

**Actions:**
1. Add to `src/scoring/constants.ts`: `export const CLAUDE_MD_EXISTS_POINTS = 15;`
2. Create `src/scoring/checks/claude-md-exists.ts`:
   ```typescript
   import { Check } from '../index';
   import { CLAUDE_MD_EXISTS_POINTS } from '../constants';
   import { existsSync, readFileSync } from 'fs';
   import path from 'path';

   export function checkClaudeMdExists(dir: string): Check[] {
     const filePath = path.join(dir, 'CLAUDE.md');
     const passed = existsSync(filePath) && readFileSync(filePath, 'utf-8').trim().length > 0;
     return [{
       id: 'claude-md-exists',
       label: 'CLAUDE.md exists and has content',
       points: CLAUDE_MD_EXISTS_POINTS,
       passed,
       details: passed ? 'CLAUDE.md found with content.' : 'CLAUDE.md missing or empty.'
     }];
   }
   ```
3. Update `src/scoring/index.ts`:
   - Add: `import { checkClaudeMdExists } from './checks/claude-md-exists';`
   - In `runChecks()`: `checks.push(...checkClaudeMdExists(dir));`
4. Create test in `src/scoring/__tests__/claude-md-exists.test.ts`
5. Run: `npm run test -- src/scoring/checks`

**Result:** New check appears in `caliber score` output with 15 points on pass.

## Common Issues

**"Cannot find module 'src/scoring/checks/check-name'"**
- Verify file exists at exact path: `src/scoring/checks/<check-name>.ts`
- Verify import in `src/scoring/index.ts` uses correct filename
- Run: `ls -la src/scoring/checks/` to list files

**"Check returns undefined or null"**
- Check function must return an array `Check[]`, never `undefined`
- Always wrap return: `return [{ ... }];` even for single checks
- Verify `return` statement exists before the closing brace

**"Check undefined in scoring output"**
- Verify `checkName()` is called in `runChecks()` in `src/scoring/index.ts`
- Verify `checks.push(...checkName(dir));` is present
- Run: `npm run test -- src/scoring` to catch registration errors

**"Test fails: 'No such file or directory'"**
- Use `memfs` and `vol.fromJSON()` to mock filesystem; do NOT use real `fs`
- Example: `vol.fromJSON({ '/project/CLAUDE.md': 'content' });`
- Verify `beforeEach(() => vol.reset());` clears state between tests

**"Points value is wrong or missing"**
- Verify constant is defined in `src/scoring/constants.ts`: `export const CHECK_NAME_POINTS = <number>;`
- Verify import in check file: `import { CHECK_NAME_POINTS } from '../constants';`
- Verify check uses imported constant: `points: CHECK_NAME_POINTS,` not hardcoded number
- Run: `grep -n 'CHECK_NAME_POINTS' src/scoring/constants.ts src/scoring/checks/<check-name>.ts` to verify both exist