AGENTS.md git:20260930.c1b179fA

Immutable. This exact content never changes and is served at /api/v1/blob/68a59822d8033c84.

# Cartwheel course repository

Cartwheel is the support agent used throughout "Evaluating and Improving AI Agents." Students complete the starter code and use the same repository for later evaluation exercises.

## Find the relevant instructions

- Read [README.md](README.md) for setup and commands. Run commands from the repository root.
- For homework help, identify the assignment from the request and existing work. If it is unclear, ask which assignment the student is working on. The [homework index](homework/README.md) lists released assignments.
- For HW1, HW2, or HW3, read the relevant handout in `homework/module-1/` and follow its requirements and deliverables. When the student invokes a handout's interactive walkthrough, follow its pacing and review points. For HW3, also read and follow `scenarios/skill/SKILL.md`; keep query generation separate from application execution and stop for the required human reviews.
- For repository maintenance, follow the requested change directly. Preserve unfinished homework functions unless implementing them is part of the request.
- For HW4, read the handout in `homework/module-2/` and the [error-discovery](https://github.com/ai-evals-course/evals-skills/blob/main/skills/error-discovery/SKILL.md) skill on GitHub. For HW5, use the current handout with [write-judge-prompt](https://github.com/ai-evals-course/evals-skills/blob/main/skills/write-judge-prompt/SKILL.md) and [validate-evaluator](https://github.com/ai-evals-course/evals-skills/blob/main/skills/validate-evaluator/SKILL.md). HW5 requires one judge; additional judges and prevalence estimation are optional. Follow its label minimums, metric calculations, and interrupted-test instructions. Do not require numerical acceptance targets. Preserve the student's human labels and final decisions.
- For HW8, read `homework/module-5/hw8.md` and the files in `optimize/`. When the student chooses the improve loop, follow `.agents/skills/improve-loop/SKILL.md` with `optimize/program.md`. Never run test cases before the student saves the final agent configuration.

## Shared rules

- Read [SPEC.md](SPEC.md) and the relevant function contracts before changing agent behavior. Policy numbers come from `facts.yaml`. Editing the specification alone does not change the running application.
- Reuse the supplied database and authorization helpers and return structured tool results.
- Preserve permission checks, refund thresholds, human approval, and kill-switch protections. Keep evaluation cases as regression tests and keep evaluation inputs out of prompts.
- Preserve existing student work and settings. Before regenerating data, check whether it would erase work the student wants to keep. Generate demo data through the seed tools and preserve the pinned demo orders. Confine adversarial fixtures to temporary database copies and keep their generated data out of commits.
- Handle API keys locally through `.env`. Never request keys in chat, print their values, or commit them. Refer to credentials by environment variable name.
- Homework placeholders intentionally raise `NotImplementedError`. Expected failures are unfinished work, not proof of completion. Follow the handout's focused tests and run relevant regression checks; resolve mismatches without weakening requirements or tests merely to pass.
- Report which checks ran offline and which used a live model, and record only observed conversations and tool results. When helping with a submission, leave the student's assessments and recording to the student, and keep unverified deliverables marked as pending.

The repository root `AGENTS.md` is the canonical instruction file. The root `CLAUDE.md` is a relative symlink to it; edit the target rather than maintaining a second copy.