hiring-scorecard · diff
git:20260705.acb5571 to git:20260705.a28c4ce
10 added, 10 removed. Audit A to A.
---
name: Hiring Scorecard Builder
description: Builds structured interview scorecards tied to role-specific signals to reduce bias and improve calibration. Use when opening a new role or auditing an existing interview loop for consistency.
---
# Hiring Scorecard Builder
Unstructured interviews have poor predictive validity and amplify the biases of whoever talks most. A scorecard forces interviewers to decide what matters before meeting the candidate and to gather evidence, not impressions.
- ## Step 1 — Define the Role's Top 4-6 Signals
+ ## Step 1 - Define the Role's Top 4-6 Signals
Start from the first 90-day outcomes, not a list of desired traits. Ask: what does success look like concretely at 30, 60, and 90 days? Then work backward to the signals a candidate would have to demonstrate to make those outcomes likely. Good signals are observable in a conversation. Bad signals are adjectives ('smart', 'passionate', 'culture fit').
- ## Step 2 — Map Signals to Interview Slots
+ ## Step 2 - Map Signals to Interview Slots
- Each interviewer owns 1-2 signals. Never have two interviewers independently assess the same signal with no calibration plan — duplication wastes time and creates conflicting data. Assign technical depth signals to engineers, cross-functional signals to PMs or peers, and leadership/judgment signals to the hiring manager.
+ Each interviewer owns 1-2 signals. Never have two interviewers independently assess the same signal with no calibration plan - duplication wastes time and creates conflicting data. Assign technical depth signals to engineers, cross-functional signals to PMs or peers, and leadership/judgment signals to the hiring manager.
- ## Step 3 — Write the Scoring Rubric
+ ## Step 3 - Write the Scoring Rubric
For each signal, write what a 1 (does not meet bar), 3 (meets bar), and 5 (exceeds bar) looks like in observable terms. Example for 'navigates ambiguity': a 1 asks for requirements to be fully specified before acting; a 3 defines scope, makes assumptions explicit, and validates them; a 5 proactively creates structure where none exists and coaches others through it. The rubric must be written before interviews start.
- ## Step 4 — Require Evidence, Not Ratings
+ ## Step 4 - Require Evidence, Not Ratings
The scorecard submission form has two fields per signal: a numeric score AND a required evidence quote from the candidate. No evidence, no score. This prevents post-hoc rationalization and makes debrief discussions concrete.
- ## Step 5 — Run a Calibrated Debrief
+ ## Step 5 - Run a Calibrated Debrief
- Collect scores independently before the debrief meeting. Start by reading scores aloud before anyone speaks. Then discuss only the signals where scores diverge by 2 or more points — those are the interesting ones. The hiring manager does not share their view first; they synthesize last.
+ Collect scores independently before the debrief meeting. Start by reading scores aloud before anyone speaks. Then discuss only the signals where scores diverge by 2 or more points - those are the interesting ones. The hiring manager does not share their view first; they synthesize last.
## Escape Hatches
For a high-volume pipeline, reduce to 3 signals and a 3-point scale. For a senior or staff role, add a written exercise component and score the artifact, not the person's delivery style.
## Deliverable
Produce a complete scorecard package: the 4-6 named signals each traced to a 30/60/90-day outcome, the signal-to-interviewer slot map, a written 1/3/5 rubric per signal in observable terms, the two-field submission form (score plus required evidence quote), and the debrief protocol (independent submission, scores read aloud first, hiring manager synthesizes last).
## Do NOT
- - Do not write the rubric after interviews begin — a bar defined mid-loop bends toward the candidates already seen, which is exactly the post-hoc rationalization scorecards exist to prevent.
- - Do not use adjective signals ('strong communicator', 'culture fit') — anything not observable in a conversation becomes a proxy for likability and similarity bias.
+ - Do not write the rubric after interviews begin - a bar defined mid-loop bends toward the candidates already seen, which is exactly the post-hoc rationalization scorecards exist to prevent.
+ - Do not use adjective signals ('strong communicator', 'culture fit') - anything not observable in a conversation becomes a proxy for likability and similarity bias.
- Do not let interviewers share impressions before scores are submitted; early opinions anchor the group and erase the independent-signal value of a panel.
- - Do not average divergent scores into a middle number — a 1 and a 5 on the same signal means the interviewers saw different things or hold different bars, and that disagreement is the data.
+ - Do not average divergent scores into a middle number - a 1 and a 5 on the same signal means the interviewers saw different things or hold different bars, and that disagreement is the data.
- Do not assign more than 2 signals to one interviewer; coverage gets shallow and the interview becomes a checklist sprint instead of an evidence-gathering conversation.
- Do not carry a scorecard between materially different roles; signals derived from one role's 90-day outcomes do not transfer to another's.
## Quality bar
- Every signal traces to a concrete 30/60/90-day outcome and is observable in an interview.
- The 1/3/5 rubric exists in writing before the first interview is scheduled.
- Every submitted score has a verbatim evidence quote attached; scores without evidence are returned, not accepted.
- The debrief discusses only divergences of 2+ points, and the hiring manager speaks last.