baseline-comparison-review · diff
git:20260502.9239fa2 to git:20260502.2a92051
13 added, 2 removed. Audit B to A.
---
name: baseline-comparison-review
- description: TODO – Phase 3 authoring pending.
+ description: "Ensure that new complex models actually outperform simple, naive baselines."
---
- <!-- PLACEHOLDER: Content to be written in Phase 3. -->
+ # Baseline Comparison Review
+ Machine learning models add massive technical debt. You must constantly justify their existence by comparing them to a "dumb" baseline.
+
+ ## Review Protocol
+
+ 1. **Define the Naive Baseline**:
+ - For classification: Predict the majority class.
+ - For regression: Predict the mean or median of the training target.
+ - For time series: Predict the last known value (naive persistence).
+ 2. **Define the Heuristic Baseline**: What simple `if/else` rule would a domain expert write?
+ 3. **Evaluate the Delta**: If the complex Deep Learning model only beats the heuristic baseline by 1%, recommend keeping the heuristic. The complexity is not worth the maintenance cost.
+ 4. **Action**: Always demand a baseline evaluation script before approving a new model architecture.