llms.txt git:20260531.a533e76A

Immutable. This exact content is served forever at /api/v1/blob/6d95357055d3fa48.

# hermes-jailbench

Deterministic jailbreak regression benchmark for LLM endpoints.

Use this repo when you need to:
- rerun known jailbreak attempts after a prompt or model change
- compare refusal rates across releases
- save markdown or JSON evidence for audit trails

Primary CLI:
- `hermes-jailbench --demo`
- `hermes-jailbench --dry-run`
- `hermes-jailbench --model <model> --api-key $ANTHROPIC_API_KEY`

Outputs:
- per-attack verdicts
- refusal, partial, and compliance counts
- category breakdown
- markdown or JSON report

Do not use this repo as:
- proof against novel jailbreaks
- a multi-turn red-team harness
- a semantic judge for edge cases without human review

Key success condition:
- the same attack corpus and response text produce the same score and report shape


## About Hermes Labs

Hermes Labs is an independent AI-reliability lab building open-source tools that catch silent failure modes in production AI. More at https://hermes-labs.ai