git:20261008.3f25f14 to git:20261015.f440cbb

- description: Use when designing prompts, evals, or LLM-powered features — prompt architecture, model selection, guardrails, and eval harnesses for prompt-as-product surfaces. Triggers on "prompt engineering", "LLM feature", "eval harness", "prompt design", "提示词工程", "LLM 特性", "prompt 设计".

- Design prompts and evals as a first-class engineering deliverable. `context-engineering`

- assembles context for a coding agent; this skill designs the prompt-as-product surface — the

- prompts, model choices, guardrails, and eval harnesses behind LLM-powered features. A prompt

- without an eval is an opinion; an eval without a prompt is a benchmark. Ship neither blind.

-

- - Designing an LLM-powered feature (chat, summarization, extraction, classification, generation)

- - Building or refining a prompt for production use

- - Creating an eval harness to measure prompt/model quality

- - Selecting a model for a specific task against cost/latency/quality trade-offs

- - Triggers on "prompt engineering", "LLM feature", "eval harness", "prompt design", "提示词工程", "LLM 特性"

- **Not for:** assembling context for a coding agent (use `context-engineering`); general research

- on a topic (use `research`); API contract design for non-LLM endpoints (use `api-design`).

- ### 1. Define the task and success criteria

-

- State the task in one sentence, then define measurable success criteria — without these, prompt

- iteration is vibes-driven. Pull from the product spec:

-

- - Input space: what inputs will the prompt receive? (vary by length, language, edge case, adversarial)

- - Output contract: structured output (JSON schema), free text, or classification?

- - Quality bar: accuracy %, format adherence %, hallucination rate, latency target, cost per call

- - Failure modes to prevent: what must the model NEVER do? (leak PII, invent facts, refuse valid input)

-

- _Verify: success criteria are written as measurable thresholds, not "good responses."_

-

- ### 2. Design the prompt architecture

-

- Structure the prompt as components, not a wall of text — each component has a job:

-

- - **System / role:** who the model is, what it must and must not do (guardrails live here)

- - **Task instruction:** the operation, stated precisely with the output format

- - **Context / retrieved data:** only the facts the model needs (not the whole knowledge base —

- context bloat degrades accuracy and raises cost)

- - **Few-shot examples:** 2–5 input→output pairs covering the happy path and an edge case; place

- before the actual input, after the instruction

- - **Output format:** explicit schema or template; use structured output (JSON mode / function

- calling) when the downstream system parses the result

-

- Separate stable parts (system, format) from variable parts (user input, retrieved context) so

- prompt caching applies to the stable prefix. _Verify: each prompt component is labeled and serves

- one purpose; no component is duplicated._

-

- ### 3. Select the model

-

- Match the model to the task's quality, latency, and cost profile — don't default to the most

- capable model for every call:

-

- - **Complex reasoning / generation:** most capable model, accept higher latency and cost

- - **Classification / extraction / routing:** smaller, faster, cheaper model — these tasks are

- over-served by frontier models

- - **High-volume, low-stakes:** the cheapest model that meets the quality bar; route edge cases to

- a stronger model (cascaded / routed architecture)

- - Evaluate multiple models against the eval harness (step 4) before committing — model choice is

- empirical, not reputational

-

- _Verify: model choice is documented with the eval comparison and the cost/latency/quality

- trade-off that justified it._

-

- ### 4. Build the eval harness

-

- An eval harness is the test suite for prompts. It runs the prompt against a labeled dataset and

- scores the output against the success criteria:

-

- - **Dataset:** 50–200 examples minimum, covering happy path, edge cases, and known failure modes.

- Use real production inputs where possible; synthesize adversarial cases for gaps.

- - **Metrics:** automated where possible (exact match, JSON schema validity, BLEU/ROUGE for

- generation, classifier-based safety); human review for quality dimensions automation can't score.

- - **Regression suite:** the eval harness runs on every prompt or model change — a prompt tweak

- that improves one case but regresses three others is a bug, not an improvement.

-

- _Verify: the eval harness runs from one command, produces a scorecard, and is committed alongside

- the prompt._

-

- ### 5. Add guardrails

-

- Production prompts need defenses the prompt itself cannot provide:

-

- - **Input validation:** reject malformed, oversized, or disallowed inputs before they reach the model

- - **Output validation:** parse and schema-check model output; reject and retry (or fallback) on

- malformed output

- - **Content filters:** safety classifiers for harmful content (input and output)

- - **PII redaction:** strip sensitive data from prompts before logging; never log raw user input

- that may contain PII to a shared prompt-logging system

- - **Rate limits and cost caps:** bound per-user and aggregate spend; a prompt bug that loops can

- burn a budget in minutes

- _Verify: every guardrail has a defined failure behavior (reject, retry, fallback) — silent

- pass-through is not a guardrail._

- ### 6. Iterate against evals

- Iterate the prompt against the eval harness — change one variable at a time (instruction,

- examples, model, temperature), re-run evals, keep the change only if the scorecard improves

- without regression. Document prompt versions and their eval scores alongside the prompt, the way

- code commits pair with test results.

- **Output:** `docs/design/PROMPT.md` — the prompt design document: task definition, success

- criteria, prompt architecture (component breakdown), model selection rationale, eval harness

- description and dataset, guardrails, and version history with eval scores. Close with an AUD

- block (6–12 items: success-criteria measurability, eval coverage of failure modes, guardrail

- failure behaviors, cost caps, PII handling) — implementation starts only after the human audit

- passes. Format and statuses: `${CLAUDE_PLUGIN_ROOT}/references/human-audit.md`.

- - [ ] Task stated in one sentence; success criteria are measurable thresholds

- - [ ] Prompt structured into labeled components (system, instruction, context, examples, format)

- - [ ] Stable and variable parts separated for prompt caching

- - [ ] Model selected via eval comparison, not reputation; trade-off documented

- - [ ] Eval harness: 50+ examples, automated metrics, runs from one command, committed

- - [ ] Guardrails on input, output, content, PII, and cost — each with a failure behavior

- - [ ] Prompt changes paired with eval re-runs; no regression accepted

- - [ ] `docs/design/PROMPT.md` produced with architecture + eval + guardrails + version history

-

- **Red flags:** a single wall-of-text prompt with no structure; choosing the most expensive model

- for a classification task; shipping a prompt with no eval; logging raw prompts with user PII; no

- output schema validation (trusting the model to always return valid JSON); prompt iteration by

- vibes without a scorecard; no cost cap on a production prompt.

- - [${CLAUDE_PLUGIN_ROOT}/references/engineering-principles.md](${CLAUDE_PLUGIN_ROOT}/references/engineering-principles.md) — shared discipline (surface assumptions, verify don't assume, goal-driven execution)

- - [${CLAUDE_PLUGIN_ROOT}/references/human-audit.md](${CLAUDE_PLUGIN_ROOT}/references/human-audit.md) — audit-block baseline: AUD vs 确认 blocks, status lifecycle, readability bar