backtest-review · git:20261012.b5abe77 · 2026-10-12 · sha256 19ad476e18fee8a1

backtest-review git:20261012.b5abe77C

Immutable. This exact content never changes and is served at /api/v1/blob/19ad476e18fee8a1.

No tool to install first
claude plugin marketplace add https://markdownregistry.com/mp/a/art_xo7qjnm3htxpgjgd/19ad476e18fee8a198a0bda3dd99caccdd3c74f1e44d877f098dbacbebb53b7d.json

Then /plugin install backtest-review@markdownregistry-xo7qjnm3htxpgjgd in Claude Code. It installs this exact version, the one graded C here, pinned to the commit it was graded at, and leaves it there.

<!--
  Imported from: https://github.com/saksham10arora-dotcom/quant-skills
  Source Path: skills/backtest-review/SKILL.md
  Curated for awesome-quant-skills
-->

---
name: backtest-review
description: Review a trading strategy or backtest for look-ahead bias, overfitting, and statistical validity. Use when the user shares backtest code, equity curves, Sharpe ratios, or asks whether a trading signal is real.
---

# Backtest Review

Review any strategy artifact in this order. Do not skip steps; most fatal flaws live in step 2.

## 1. Inventory before judging

Identify: universe, rebalance frequency, signal definition, execution assumptions, cost model, in-sample vs out-of-sample split, number of variants tried. If any of these are unstated, ask before analyzing.

## 2. Look-ahead sweep (kills most strategies)

Check every data access for information that would not exist at decision time:
- Full-sample normalization (z-scores, min-max) computed across the whole history
- Signals using close prices of day t to trade day t
- Fundamentals joined without point-in-time availability dates
- Survivorship: universe built from today's index membership
- Targets shifted the wrong direction (`df.shift(-1)` used as feature instead of label)

Report each finding with file and line references.

## 3. Multiple testing honesty

Ask: how many total variants were tried before this one? Apply these rules of thumb:
- Under ~5 trials: judge the raw statistics
- 5-50 trials: require out-of-sample confirmation on data never touched during iteration
- 50+: demand explicit deflation analysis (deflated Sharpe ratio, Bailey & Lopez de Prado); a Sharpe under 1 is presumptively noise at this search scale

## 4. Distribution sanity

Compute and report: skewness, kurtosis, max drawdown, longest drawdown duration, lag-1 autocorrelation of returns.
- |autocorrelation| significant at sqrt(n) > 2 → suspect smoothed marks or stale fills
- Kurtosis > 6 → warn that Sharpe understates tail risk
- Win rate > 65% with positive skew on daily bars → check for hidden short-vol exposure

## 5. Verdict format

Deliver exactly:
```
VERDICT: REAL / UNPROVEN / LIKELY ARTIFACT
Fatal issues: [list or none]
Statistics: [Sharpe, PSR-style confidence note, DD]
Next test: [single highest-value next action]
```

Be direct. A kind lie about a strategy costs more than a harsh truth.