v1.2.1 to v1.2.2

---

name: research-loop

author: Personify Labs

description: >

Orchestrates survey-architect, feedback-synthesizer, and research-reporter

as an iterative research loop — not a one-way pipeline. Detects when a

downstream skill can't proceed cleanly (data too thin to trust, wrong

instrument for the stated goal) and routes back upstream instead of

producing a shaky output. Maintains an explicit per-study state

(00-state.json, against a fixed transition graph — not a full workflow

engine) alongside the per-study file structure at

/research/<study-name>/ and the per-product longitudinal benchmark at

/research/_benchmarks/<product>.md, and checks benchmark comparability

(same instrument/wording/scale/population/sampling/trigger) before

letting two waves imply a trend. Use this as the single entry point

for any UX research task — a new study, raw data to make sense of, or a

report to write — rather than calling the three skills directly. Trigger

when someone starts a new research study, pastes raw feedback/survey

data without a study already in motion, or asks to "run the research

loop." Bundles scripts/routing.py for deterministic entry detection,

state transitions, the loop-back ceiling, and benchmark validation (run

it, don't count loop-backs or eyeball comparability by hand) and evals/

for regression testing.

tags:

- user-research

- ux-research

- orchestration

- router

license: CC-BY 4.0

---

# Research Loop

Runs survey-architect, feedback-synthesizer, and research-reporter as a

loop that behaves like an actual research team: someone can say "that

sample's too small, we need more data" and send it back, rather than

quietly writing up a shaky number.

Each of the three skills works standalone. This skill exists for the

handoffs between them and the file structure that lets a later skill (or

a later *study*) pick up where an earlier one left off.

-----

## The loop at a glance

```

Entry detection

↓

survey-architect (intake → instrument → sample size → deploy-ready spec)

↓

[ pause: wait for data collection ]

↓

feedback-synthesizer (score + CI + significance + theme + severity)

↑___________kick back if n insufficient for stated goal_______|

↓

research-reporter (audience-formatted output)

```

Max loop-backs: 2 per study. If a second kickback doesn't resolve it,

surface the trade-off to the researcher rather than looping a third time.

-----

## Entry detection

**Run `scripts/routing.py`'s `detect_entry_point(...)`** rather than

eyeballing the table below — pass whether a learning goal, raw data, an

existing study folder, and a synthesis file are present, and use its

return value. The table is here to show the reasoning, not to be

hand-matched each time:

| What's pasted / asked | Entry point |

|---|---|

| A research question, no data yet ("I need to know if X is usable") | survey-architect |

| Raw survey results / CSV / feedback pile, no study folder exists yet | feedback-synthesizer (standalone mode — no spec file to read) |

| Raw data + an existing `/research/<study-name>/` folder | feedback-synthesizer (loaded mode — reads the spec) |

| A finished `02-synthesis.md` and a request to write something up | research-reporter |

| "Run the research loop" with a vague ask | Start at survey-architect Step 0 (intake) — let intake clarify |

A finished synthesis file always wins the routing decision, even if a

learning goal or raw data is also present — it's the most-progressed

artifact for that study. Don't force every input through all three

stages: a pile of app store reviews with no survey behind it goes

straight to feedback-synthesizer in standalone mode, full stop — this

is a common failure mode worth guarding against explicitly (see

qualitative_cases.md Case 1).

`detect_entry_point` is the **cold-start** path — it infers progress by

checking which files exist, for a study that has no state file yet

(including one that predates this state model). Once `00-state.json`

exists for a study, prefer reading it (below) over re-sniffing files on

every turn.

-----

## File structure

```

/research/<study-name>/

00-intake.md — product, learning goal, instrument decision (survey-architect)

00-intake.json — same facts, canonical/machine-readable (survey-architect)

00-state.json — explicit state + loopback count + artifact validation (research-loop)

01-survey-spec.json — deploy-ready spec: items, scoring, n, triggers (survey-architect)

02-synthesis.md — scores, CIs, significance, themes, severity (feedback-synthesizer)

02-synthesis.json — same facts, canonical/machine-readable (feedback-synthesizer)

03-report-internal.md — internal-audience report (research-reporter)

03-report.json — same facts, canonical/machine-readable (research-reporter)

03-report-formal.docx — formal-audience report, if requested (research-reporter)

/research/_benchmarks/<product>.md

— running log: date, instrument, score, CI, n, measurement conditions, per wave

— running qual theme taxonomy for the product

```

`research_root` (`/research`) and `benchmarks_root`

(`/research/_benchmarks`) come from `../config/product-context.yaml` —

`study_dir()`/`benchmark_path()` default to those same values, but the

config file is the place to change them, not this file or the scripts.

For every `.md`/`.json` pair above, **the `.json` is canonical** — it's

what the next skill reads and what state/validation logic checks. The

`.md` is the human-readable copy of the same facts, never a place where

additional information lives that the `.json` doesn't have.

Every skill reads what exists before assuming it needs to build it. If

`00-intake.md`/`.json` already answers a question a skill would

otherwise ask the researcher, use it — don't re-ask.

-----

## State

`00-state.json` shape: `study_id`, `state` (one of `routing.STATES`:

`INTAKE`, `SURVEY_DESIGN`, `DATA_READY`, `SYNTHESIS`,

`SYNTHESIS_REVIEW`, `REPORTING`, `REPORT_REVIEW`, `COMPLETE`),

`loopbacks` (`{total, max}`), `artifacts` (artifact name → path),

`validation` (artifact name → `"valid"` or the problem list a

`validate_*` function returned).

**Read/write it through the script, not by hand:**

- `routing.read_state(study_dir_path)` — `None` if the study hasn't

started tracking state yet (fall back to `detect_entry_point`).

- `routing.next_state(current_state, target_state)` — validates the

transition against a fixed graph before it happens; raises (naming the

actually-allowed next states) on an invalid jump, e.g. `INTAKE`

straight to `REPORTING`. Allowed kickback edges: `SYNTHESIS →

SURVEY_DESIGN`, `SYNTHESIS_REVIEW → SYNTHESIS`, `REPORT_REVIEW →

SYNTHESIS`.

- `routing.write_state(study_dir_path, state)` — persists it.

This is deliberately a flat file plus a small fixed graph, not a

workflow engine — its only job is to catch an accidental stage-skip and

to replace re-sniffing files on every turn with reading one file. It

doesn't replace judgment about *when* a stage is actually done; a skill

still decides that before calling `next_state`.

Print this at every handoff so the researcher can see where things stand

(the state block below can be derived directly from `00-state.json`):

```

━━ Research Loop State ━━━━━━━━━━━━━━━━━━━━

Study: [study-name]

Product: [product, from intake]

Learning goal: [one line, from intake]

State: [current STATES value]

Stage:

✓ survey-architect instrument: [X], n required: [Y]

● feedback-synthesizer in progress

○ research-reporter pending

Kickbacks: [none | count + reason]

Confidence: [none flagged | LOW — n=X vs required Y]

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

```

-----

## Kickback conditions

A kickback is any point where a downstream skill cannot honestly proceed

without an upstream decision changing.

**feedback-synthesizer → survey-architect:**

Sample size is so far below the required floor that the stated learning

goal genuinely can't be answered (not just "flag and proceed" — that's

the normal small-n handling inside synthesizer itself; this is the

escalated case where the gap is too large to paper over with a wider CI).

Kick back with: current n, required n, and a recommendation (extend

collection window, widen distribution, or accept a narrower claim than

originally scoped).

**feedback-synthesizer → survey-architect (instrument mismatch):**

The data doesn't actually support the learning goal the intake recorded

— e.g., a single NPS item was collected but the real question was about

task-level usability. Kick back with what instrument should have been

used, so the next wave corrects it.

**research-reporter → feedback-synthesizer:**

The synthesis file is missing something the report structure requires

(no severity tiers, no CI on a headline score) — this should be rare if

synthesizer followed its own spec, but the reporter should never

backfill or invent a missing CI/severity itself.

Output format for any kickback:

```

━━ Kickback ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

From: [skill]

To: [skill]

Reason: [one sentence]

What changes: [what the upstream skill should do differently this time]

Loop-back count: [N of 2 max]

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

```

-----

## Max loop-backs

**Run `scripts/routing.py`'s `should_escalate_to_researcher(loopback_count)`**

before attempting another loop-back — don't just count in your head.

If it returns `should_escalate=True` (2 loop-backs already used):

> "Two loop-backs on [issue] without resolving it. Here's where things

> stand: [current state]. Options: A — accept the limitation and report

> it as a caveat, B — pause the study and revisit scope. Which do you

> want?"

Don't attempt a third loop-back without the researcher's input.

-----

## Modes

**Gated (default):** pause at each stage transition and each kickback

for confirmation. Use for a new study or an unfamiliar product.

**Fast:** run straight through, noting any kickbacks in the final

summary rather than pausing on them. Opt in with "fast mode" or "just

run it."

-----

## Loop completion block

```

━━ Research Loop Complete ━━━━━━━━━━━━━━━━━

Study: [study-name]

Product: [product]

Instrument: [instrument used]

n: [final n] Confidence: [flagged / not flagged]

Headline result: [score + CI, one line]

vs. benchmark: [trend direction if routing.check_benchmark_comparability

says comparable — otherwise "not comparable: <mismatched

fields>", waves reported side by side instead of a trend]

Top findings: [Critical-tier items, one line each]

Kickbacks: [none | list with reason]

Outputs:

Internal report: [path]

Formal report: [path, if generated]

Benchmark file: updated at /research/_benchmarks/<product>.md

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

```

-----

## Rules

Priority when rules interact: **STOP** > **MUST NOT** > **MUST** >

**SHOULD** > **MAY** — see `ux-research/README.md` for the full

precedence explanation (that file is the one place it's spelled out;

every skill's Rules section just tags bullets with these keywords).

- **STOP** before a third loop-back — `should_escalate_to_researcher`

returning `True` means end processing and surface the trade-off

(options A/B below), not attempt the loop-back anyway.

- **MUST NOT** force standalone feedback (no survey behind it) through

survey-architect first.

- **MUST NOT** treat a synthesizer small-n flag as an automatic

kickback — that's normal handling inside synthesizer; only an

unanswerable-goal gap escalates to a loop kickback.

- **MUST NOT** call `next_state` with a jump the transition graph

doesn't allow, and MUST NOT advance state without running the

validator for the artifact that stage produced.

- **MUST NOT** let research-reporter fill in a missing CI or severity

tier itself instead of kicking back to synthesizer.

- **MUST NOT** imply a benchmark trend when

`check_benchmark_comparability` reports a mismatch — report the waves

side by side instead.

- **MUST** write to `/research/_benchmarks/<product>.md` after a

completed synthesis.

- **SHOULD** skip re-running survey-architect's intake when `00-intake.md`/

`.json` already answers the same questions.

-----

## Quick reference

| Input | Entry point |

|---|---|

| New research question, no data | survey-architect |

| Raw data, no study folder | feedback-synthesizer (standalone) |

| Raw data, study folder exists | feedback-synthesizer (loaded) |

| Synthesis done, need a writeup | research-reporter |

| Synthesizer: n too low for the goal | Kick back to survey-architect |

| Synthesizer: wrong instrument for the goal | Kick back to survey-architect |

| Reporter: synthesis missing CI/severity | Kick back to feedback-synthesizer |

| 2 kickbacks, still unresolved | Surface to researcher, stop |

| Two waves, measurement conditions differ | Report side by side, not as a trend |

| Loop complete | Completion block, update benchmark file |