CLAUDE.md git:20260820.94b61fbA

Immutable. This exact content never changes and is served at /api/v1/blob/25de023b505a9f29.

# ISA Toolkit — Claude Code Context

## Repository
- **Repo:** `andremun/InstanceSpace` (MATLAB, R2025a+)
- **Owner:** sole developer (Mario Andrés Muñoz) — no human reviewers; GitHub Copilot is the automated PR review gate. `master` has branch protection requiring changes go through a PR.
- **Primary API:** the `InstanceSpace` class (`InstanceSpace.m`) — `build()`/`explore()`/`save()`/`InstanceSpace.load()`/`getResults()`/`plot()`. `buildIS.m`/`exploreIS.m` are thin backward-compatibility wrappers around it (kept for callers like the MATILDA web platform); don't add new logic to them directly.
- **Completed refactor plan:** `docs/tech/isa_refactor_plan_v1.7.pdf` — historical source of truth for the v0.9.0 refactor (all ten phases). Now fully shipped; treat it as an archival reference, not a live backlog.
- **Current backlog:** GitHub Issues, milestone `v0.9.1` — this is the live source of truth for what's next, not the old plan document. As of the last audit, everything in it is closed except #30 (deferred to v0.9.2); the CLOISTER-3D-hull follow-up is filed as #50 but not milestone-assigned (also deferred to v0.9.2 in spirit). See "Current state" below for the shape of it.
- **Reference papers:** `docs/papers/Smith-Miles2023.pdf` (general ISA methodology), `docs/papers/Simpson2025.pdf` (ISA3, the 3D extension — cited only in `PILOT.m`, `PILOTviewpoint.m`, `TRACE.m`, and the 3D-output files in `output/`, not repo-wide). PILOT's origin (Muñoz et al. 2018), legacy TRACE's origin (Muñoz & Smith-Miles 2017), and FILTER's origin (Alipour et al. 2023) are cited in each file's own docstring `Reference:` block but aren't archived as PDFs here.

## Test harness
Run from the repo root; `buildIS.m`/`exploreIS.m` print `EOF:SUCCESS` on success (errors abort before printing), and `test_integration.m` prints `EOF:SUCCESS`/`EOF:ERROR` at the end:
    example
    test_integration
`example.m` already runs the full `buildIS`+`exploreIS` pipeline internally on the bundled reference dataset — it's the one to start with. `test_integration.m` is a thin `matlab.unittest`-based runner (#39) over `tests/*.m` (`PipelineOptionsTest` — every classifier, tuning strategy, 2D/3D, PLS, viewpoint groups, parameterised; `ClassApiTest` — staged `build()`/`explore()`/save-load round-trips; `MigrationTest` — the full `ISAmigrateModel` legacy-migration table; `RegressionTest` — targeted fixed-bug regressions), with a `CodeCoveragePlugin` producing `coverage.xml`. New features and bug fixes get a test *method* under `tests/`, not a new block in `test_integration.m` itself. Good as a reference for how an option is meant to be used, not the place to start reading. Runs automatically on every push/PR via CI (#34, `.github/workflows/tests.yml`).

## Current state (v0.9.0 shipped; v0.9.1 in progress)
- **All ten refactor-plan phases shipped as v0.9.0** (July 2026): rewritten PRELIM/SIFTED/PILOT/PYTHIA/TRACE pipeline, the `InstanceSpace` class API, `ISAmigrateModel`/`ISAvalidateOpts`, 3D projection/viewpoint support, and a licence change from GPL v3 to PolyForm Noncommercial 1.0.0. See `RELEASE_NOTES.md` for the full changelog.
- **v0.9.1 is next**: engineering-quality, architecture, and infrastructure follow-ups only — explicitly **no algorithmic changes, no new user-facing options**. Don't expand scope on one of these issues into a feature change without checking with the user first. Highlights:
  - Remaining in scope: none currently — CLOISTER's own 2D-only convex hull for 3D projections (spun out of #32, filed as #50) is out of v0.9.1 scope (algorithmic change) and not milestone-assigned; see below.
  - Already landed on `0.9.1/development` and closed on GitHub (each closing comment cites the fixing commit; the code itself only reaches `master` once a PR is opened and merged — closing the issue tracked the fix landing, not the merge): `SECURITY.md`/`CONTRIBUTING.md` (#36), the ensemble-hyperparameter documentation closure (#33), FILTER's `isDissimilar`/`isVISA` outputs kept for diagnostics (#43), seed threading through PILOT/SIFTED (#41), the `ISAmigrateModel` LIBSVM-retraining `Yraw` fix (#42), CLOISTER's non-mean-centred-data warning (#44), LIBSVM MEX-file provenance (#29 — resolved by removal; see below), CI (#34), per-stage contract validation (#28), a shared `INIT.m` for data ingestion with PRELIM given its own train/eval dual mode (#38, closed #25/#37 as duplicates of it), `test_integration.m` migrated to `matlab.unittest` (#39), the `traceAlphaBoundary` multi-region fix (#31), CLOISTER boundary display for 2D projections (#32, option (a) — see below), and per-stage `build()`/`explore()` inspection callbacks (#26/#27).
  - **Deferred to v0.9.2** (not started): PILOT projection-orientation canonicalisation (#30) — its proposed rotation post-process and new `opts.pilot` toggle are exactly the kind of algorithmic/user-facing change this milestone excludes, even though the issue itself is reasonable on its own merits. Left open on GitHub with a comment recording the deferral; still needs its milestone field moved by hand (no milestone-management tool available in this environment). Same situation for #50 (CLOISTER's 2D-only hull for 3D projections, below) — also algorithmic, also unassigned pending a v0.9.2 milestone existing.
  - **Closed as not planned**: a cross-repo reference-fixture export tool for the Python port (`pyInstanceSpace`, #35) — per a standing preference, anything dealing with the Python port is handled from that repo, not this one, and a more streamlined approach was reportedly already underway there.
  - **Closed as resolved, no code change**: #45 (verify `PRELIM.m`'s `boxcox()` call) — confirmed directly (`which boxcox` in MATLAB R2025a) that it resolves to the Financial Toolbox's `boxcox.m`, the same dependency CI already installs for this call site (#34's commit history); combined with the already-verified positive-shift guard before each call, no gap was found.
- **Repository layout**: `core/` (INIT, PRELIM, SIFTED, PILOT, PILOTviewpoint, CLOISTER, PYTHIA, TRACE, TRACE_legacy, FILTER), `output/` (scriptcsv, scriptpng, scriptweb, scriptfcn, ISArecallView), `utils/` (ISAdefaults, ISAvalidateOpts, ISAgetClassifierFcn, ISAmigrateModel, ISAsubsetData), `deprecated/` (PYTHIA2, PYTHIAtest, SIFTED2 — warn-and-forward shims kept for backward compatibility only; don't build new code on these).

## Workflow
- Work on feature branches; merge to `master` after Copilot review. `master`'s branch protection requires a PR — don't push directly to it.
- Fix root causes, not symptoms — no `real()` wrappers, no `isfinite()` guards.
- For v0.9.1-era work, cite the relevant GitHub issue number in commit messages (e.g. `#38`), not a refactor-plan section — the plan document is closed out; issues are the live backlog now.

## Key conventions
- Language: MATLAB; style follows standard MATLAB conventions.
- No magic numbers — use named `opts` fields with documented defaults (`ISAdefaults.m`), not hardcoded literals.
- Standalone pipeline functions (`PRELIM`, `SIFTED`, `PILOT`, `CLOISTER`, `PYTHIA`, `TRACE`, `FILTER`) must remain callable independently of the `InstanceSpace` class — deliberate, not an oversight, to preserve MATILDA web-platform compatibility. Don't fold them into class methods.
- **Train vs. eval mode is dispatched via `nargin`, not a fixed positional argument**: an optional trailing `trainedModel`/`trainedTrace`/`trainedPrelim`-style struct as the *last* argument selects eval mode over train mode (e.g. `PYTHIA(Z,Y,Ybin,Ybest,algolabels,opts)` trains; `PYTHIA(...,opts,trainedModel)` evaluates; same shape for `TRACE` and, since #38, `PRELIM`/`INIT`). Confirmed by direct audit (see #38) that `SIFTED`/`PILOT`/`CLOISTER`/`FILTER` never recompute anything at explore time — they just reuse a stored trained artifact directly (e.g. `PILOT`'s trained projection is reused via a plain `X*A'` multiply) — so they don't need this structure and shouldn't get it added speculatively.
- **Data ingestion (#38)**: `INIT.m` (`core/`) does the CSV-read + `opts.selvars.feats`/`.algos` filtering that used to be duplicated between `InstanceSpace.runPrelim` and `InstanceSpace.evaluateTestSet` — `INIT(rootdir, opts)` trains (reads `metadata.csv`), `INIT(rootdir, opts, trainedModel)` evaluates (reads `metadata_test.csv`, validates the feature set/order against `trainedModel`, reconciles algorithm columns). `PRELIM.m` itself now also has the `nargin` dual mode: its Ybin/Ybest/P/beta computation (including tie-breaking) is unconditional shared code, only the bound-clipping/Box-Cox+Z-score normalisation branches on train (fit) vs eval (apply trainedPrelim's already-fit parameters) — this is what closes #37 (explore-time tie-breaking previously silently diverged from build-time, via two independently-drifted implementations). #25 and #37 are superseded by #38, not implemented separately.
- `opts.pilot.topoWeight` — still reserved/disabled, no effect in the current version; don't wire it up without confirming it's actually in scope for the task at hand.

## Known deferred items (tracked, not stale — check the linked issue before touching)
- **CLOISTER's boundary computation is 2D-only, even for a 3D projection** — `core/CLOISTER.m`'s `Zedge`/`Zecorr` use `convhull(Z(:,1),Z(:,2))` (the 2-argument form), silently ignoring the 3rd coordinate whenever `opts.pilot.dims==3`, with no warning. Found while resolving #32 (below); deliberately spun out rather than fixed as part of it, since changing what `CLOISTER.m` computes is an algorithmic change (out of v0.9.1 scope per the note above) whereas #32 itself was just about *displaying* the boundary CLOISTER already computes. `InstanceSpace.plot('boundary')`/`scriptpng.m`'s new boundary PNG both explicitly refuse a 3D projection (`ISA:InstanceSpace:boundaryNot3D`) rather than render a misleading 2D-plane outline. Filed as #50; not milestone-assigned (algorithmic change, so not v0.9.1 — see the note above; suggest v0.9.2 once that milestone exists).

Resolved, kept here as historical closure record since both were on the old plan's "deferred to Phase 9" list and Phase 9 shipped in v0.9.0 without either actually being fixed:
- `traceAlphaBoundary` (now in `output/scriptfcn.m`, moved from `scriptcsv.m` for testability) silently exported only the first region's boundary for a multi-region alpha shape — its own comment used to admit it "works correctly for simple (single-region) connected alpha shapes" only. Fixed (#31) by tracing each `boundaryFacets(poly, regionID)` region independently instead of one adjacency graph over the combined multi-region edge list; regions are NaN-separated in the output, the same convention `polyshape.Vertices` already uses.
- CLOISTER's boundary was computed but never rendered in automated output. Fixed for 2D (#32, option (a) — the 3D case is the still-open item above): `drawBoundary` in `scriptfcn.m`, a `distribution_boundary.png` in `scriptpng.m`, and an `InstanceSpace.plot('boundary')` case.

Not still open, despite also appearing on the old plan's deferred list — closed by v0.9.0 but not previously recorded as such (#33, now documented in `RELEASE_NOTES.md`): the "ensemble hyperparameter API" question (verifying `fitcensemble`'s `OptimizeHyperparameters` support). PYTHIA's shipped design tunes every classifier, including `'ensemble'`, through one shared Sobol/Bayesian layer instead of `fitcensemble`'s own optimiser, sidestepping the question rather than answering it directly — don't re-open this as if it were still pending.

**LIBSVM MEX-files removed (#29).** `svmpredict.mexw64`/`svmtrain.mexw64` were precompiled binaries with no corresponding source in the tree — removed entirely rather than kept-with-provenance, since LIBSVM is already fully deprecated for new runs. `PYTHIA`'s eval mode now raises a clear `ISA:PYTHIA:noLibsvm` error (naming the algorithm and pointing to the official LIBSVM project) if it ever needs to dispatch to a legacy LIBSVM-format classifier struct and `svmpredict` isn't on the path, instead of MATLAB's generic undefined-function error. Don't re-add these binaries; if LIBSVM support is ever needed again, get it from upstream rather than re-committing a precompiled MEX-file with no visible build source.

## `.gitignore` notes
- `docs/**/*.pdf` — tracked (negation rule), recursive under `docs/`.
- `test/data/*` — ignored except `metadata.csv`, `metadata_test.csv`, and `workspace_test.mat` (an explicit three-file allowlist). Do not broaden this to `!test/data/**` — that was tried and reverted because it makes every generated test-run output file (model.mat, CSVs, PNGs, per-case subdirectories from `test_integration.m`) trackable.