DESIGN.md@research/hint-evalA
DESIGN.md@research/hint-eval from anionex/agent-vision-toolkit is a DESIGN.md file: the design rules coding agents follow in that repo.
mdr add anionex/agent-vision-toolkit/research/hint-eval/DESIGN.md@git:20260805.c3ce904Pins git:20260805.c3ce904 in mdr.lock. Pin the fingerprint instead.
- Audit grade
- A, 14 of 14 checks passed
- Kind
- DESIGN.md file
- Repository
- anionex/agent-vision-toolkit
- Path
research/hint-eval/DESIGN.md- Versions recorded
- 5
- Latest version
- git:20260805.c3ce904, committed Aug 5, 2026
- First version recorded
- Aug 4, 2026
- Size
- 14,636 bytes
- Repository stars
- 1,243
- License
- MIT
Collected from public GitHub. Every version is saved exactly as it was and never changes. Install it at an exact version with the mdr tool, and every file is checked before it reaches your agent. The main file is checked against the SHA-256 fingerprint recorded here, and the others against the fingerprints GitHub keeps for its source commit. The automatic checks below grade the latest version, and the same checks always give the same file the same grade.
What the file says
# focus hint 效果评测 — 设计与预注册 2026-08-04 · 小规模 pilot 本文在**看到任何结果之前**写完并提交。预测写在下面，跑完不改。 目的是让「hint 有效」这句话第一次有可证伪的形式。 > 同日修订一次：初稿描述的是一套另起炉灶的 n=36 题库，而实际要跑的是既有的 > 130 题评测框架。修订把条件、题库、指标全部对齐到真正会执行的那套代码， > 并据此重写了分类预测。修订发生在五条件实验产出**第一行数据之前**—— > 当时实验处于停止状态。此后不再改预测。 > > 同日第二次修订：拆出 `contract_noverbatim`，把逐字转写条款单独隔离成一格 > （见「六个条件」与「逐字转写条款」两节）。原设计里 `naive → contract` > 同时改了角色契约和转写指令，两者的贡献无法分开。修订同样发生在 > **产出第一行数据之前**，实验尚未起跑。 > > 同日第三次修订：加一套 `captionqa_open` 条件与一条剔除规则（见文末 > 「开放式改造」一节）。**这次修订发生在看到 130 题四选一结果之后**， > 与前两次性质不同，必须如此标注：它改的是仪器灵敏度，不是假设， > 四选一那套结果原样保留报出，不被替换。 ## 追加预注册（2026-08-04，四选一结果出来之后） 四选一那 130 题跑完是零结果：全部条件落在 92%–96%，主判断 `contract → hint` 4 胜 3 负 p=1.000。原因是天花板效应——130 题里只有 15 题被任一条件答得不同， 四个选项让答题模型只需排除三个而不需要描述里真的含有答案。 改造：同图、同问题、同标准答案（`gold_text` 早已在题库里，不新增任何标注， 不由任何模型生成标准答案），去掉选项改开放式，记为数据集 `captionqa_open`， 题号加 `_open` 后缀，**与四选一结果分开报，绝不合并配对集**—— 四选一那半已饱和，合进来只会用一堆必然打平的题稀释效应量。 **剔除规则（在看到 document 25 题之后、其余 75 题产出之前写下）**： 若某题**五个条件全错，且 `native` 也错**，则视为该题去掉选项后本身有歧义， 从配对集中剔除并单独列出。依据是 `native` 直接看着图作答—— 一道连看着图都答不出的题，测的不是描述通道，是题目措辞。 document 25 题里有 3 题触发（`doc_014_q0` / `doc_105_q0` / `doc_106_q0`）， 四选一同一批题只有 1 题。**剔除前后的数字都要报**，不允许只报剔除后的。 **预注册的预期**：开放式能不能救这套评测，我判断**大概率不能**。 …
Read the whole file at its exact version.
How to install
mdr add anionex/agent-vision-toolkit/research/hint-eval/DESIGN.md@sha256:f59429ea7617d1a7The line at the top of this page pins the label git:20260805.c3ce904, which is what most people want. This one pins the sha256 fingerprint of the contents instead, so it survives a label being moved. Either way the exact version is written to mdr.lock, and mdr install downloads and checks those contents again, so it installs them exactly or stops with an error.
[](https://markdownregistry.com/a/art_rfas6vx5a36rqrzv)
3 badge image fetches in 30 days. What the badge shows.
Versions
The registry has recorded 5 versions of this file, between Aug 4, 2026 and Aug 5, 2026, more than one a day.
| version | committed | commit | size | audit | |
|---|---|---|---|---|---|
| git:20260805.c3ce904 latest | 2026-08-05 | c3ce904 | 14,636 B | A | view · diff |
| git:20260804.bf2b961 | 2026-08-04 | bf2b961 | 14,642 B | A | view · diff |
| git:20260804.803867c | 2026-08-04 | 803867c | 12,461 B | A | view · diff |
| git:20260804.c0e8dae | 2026-08-04 | c0e8dae | 11,754 B | A | view · diff |
| git:20260804.506e72e | 2026-08-04 | 506e72e | 9,049 B | A | view |
Audit of the latest version
- pass: Size between 200 bytes and 200 KB (14636 bytes)
- pass: No zero-width or bidi control characters
- pass: No instruction hidden inside an HTML comment
- pass: No link to an exfiltration or paste host
- pass: No credential-shaped string
- pass: No instruction to send local credentials anywhere
- pass: No text hidden with inline styles
- pass: No prompt-injection phrasing
- pass: No curl or wget piped into a shell
- pass: No recursive delete of root, home or parent
- pass: No instruction to read or print local credentials
- pass: No base64 blob over 200 characters
- pass: No link to a raw IP address
- pass: No script tag
Source
anionex/agent-vision-toolkit · 1,243 stars · license MIT · pushed 2026-10-13 · branch main
API
GET https://markdownregistry.com/api/v1/artifacts/art_rfas6vx5a36rqrzv GET https://markdownregistry.com/api/v1/resolve?ref=anionex/agent-vision-toolkit/research/hint-eval/DESIGN.md GET https://markdownregistry.com/api/v1/blob/f59429ea7617d1a783459fb71bea3be94e6aac933c90b0b7e80f5d6863f7fb1e
Your AI assistant does the legwork. You hear about the deals worth your word. Hand yours the standing instructions at modelranch.com and it joins the network that reads files like this one.
Guides: what DESIGN.md is and how agents use it · which agent file does what.