subject-subtyping · diff
git:20260825.cbd02ff to git:20260914.cfc1dd5
51 added, 3 removed. Audit A to A.
---
name: subject-subtyping
description: "Use this skill whenever the user wants unsupervised disease subtyping, patient stratification, latent phenotype discovery, cluster stability analysis, or low-dimensional embeddings from neuroimaging features. It supports K-means, Gaussian mixture models, spectral clustering, NMF, consensus clustering, PCA embeddings, and autoencoder embeddings. Triggers include 'subtype', 'patient stratification', 'clustering', 'latent phenotype', 'consensus clustering', 'GMM', 'NMF', 'PCA embedding', and 'autoencoder clustering'."
license: MIT
layer: base
skill_type: model
dependencies:
- statistical-ml
- run_models
---
# Subject Subtyping Skill
## Overview
`subject-subtyping` discovers unsupervised subject groups from tabular imaging or
multimodal features. It exports subtype assignments, latent embeddings,
- silhouette diagnostics, and a reusable checkpoint.
+ silhouette diagnostics, and a fitted estimator checkpoint. The reporting route
+ also exports patient-level QA, feature profiles, group sizes and figures.
+ Research use only: clusters are not clinical diagnoses. The legacy training
+ checkpoint does not bundle the training imputer/scaler and cannot by itself
+ assign new patients from raw features. Preserve the fitted preprocessing and
+ feature order in a separately validated deployment pipeline before future-patient
+ inference; spectral/consensus training does not define an out-of-sample rule.
+
**Supported models**
| Model | Method | Typical use |
|---|---|---|
| `kmeans` | Euclidean partitioning | compact baseline |
| `gmm` | Gaussian mixture | soft distributional subtypes |
| `spectral` | graph spectral clustering | non-convex structure |
| `nmf` | non-negative embedding + K-means | parts-based phenotypes |
| `consensus` | bootstrap co-clustering | stability-focused analysis |
| `pca` | PCA embedding + K-means | linear latent subtypes |
| `autoencoder` | neural embedding + K-means | nonlinear latent subtypes |
Outcome labels must not be used to choose the number of clusters. Clinical
outcomes may be tested only after subtype definitions are frozen.
---
## Installation
```bash
pip install numpy pandas scipy scikit-learn joblib torch
```
Verify:
```bash
python -c "import sklearn, torch; print('Subtyping models OK')"
```
---
## Workflows
### 1. Prepare features
Input is a CSV with `subject_id` and numeric features. Do not include diagnosis,
survival, or treatment outcome columns among the clustering features.
```text
subject_id,roi_001,roi_002,network_fc,brain_age_gap
sub-001,0.12,-0.04,0.31,2.1
sub-002,0.08,-0.09,0.27,-1.4
```
### 2. Consensus clustering
```bash
python skills/subject-subtyping/scripts/train_reference.py \
--features features.csv \
--subject-col subject_id \
--model consensus \
--n-clusters 3 \
--seed 123 \
--output-dir run_models_output/subtyping_consensus
```
### 3. PCA or autoencoder embeddings
```bash
python skills/subject-subtyping/scripts/train_reference.py \
--features features.csv \
--model pca \
--n-clusters 4 \
--latent-dim 8 \
--output-dir run_models_output/subtyping_pca
```
```bash
python skills/subject-subtyping/scripts/train_reference.py \
--features features.csv \
--model autoencoder \
--n-clusters 4 \
--latent-dim 8 \
--epochs 200 \
--output-dir run_models_output/subtyping_ae
```
### 4. Select the number of clusters
Run a prespecified range such as `k=2..8`, compare silhouette and bootstrap
stability, then freeze `k` before association with clinical endpoints. Repeat
the final model across seeds when cluster stability is central to the claim.
+ ### 5. Export the requested patient outputs
+
+ Use frozen assignments from the training route above, or an existing study; do
+ not retrain just to produce a report. Provide `feature_columns.json`, an explicit
+ JSON array of the prespecified imaging/multimodal features. Exclude IDs, site
+ identifiers and outcome/diagnosis variables from the feature set. Input feature
+ IDs must be unique, and assignments must have exactly the same subject set;
+ the report joins by ID, never by assumed row order.
+
+ ```bash
+ python skills/subject-subtyping/scripts/report_subtypes.py \
+ --features features.csv --feature-columns feature_columns.json \
+ --assignments run_models_output/subtyping_consensus/predictions.csv \
+ --output-dir run_models_output/subtyping_report
+ ```
+
+ Outputs include `subtype_assignments.csv`, `subtype_profiles.csv`,
+ `subtype_counts.csv`, `embedding.csv`, `subtype_report.png`, `report.md`,
+ `qc.json`, descriptive preprocessing parameters and a verified manifest. Profiles
+ retain observed-value missingness. Standardization and fallback PCA are
+ **descriptive**, not a new trained subtype definition or evidence of replication.
+ There are no fabricated outcome associations or subtype confidence probabilities.
+
+ If completed resampling/seed runs exist, repeat `--replicate predictions_seed2.csv`
+ to add permutation-invariant adjusted Rand and optimally matched cluster Jaccard
+ scores in `stability.csv`. These runs must cover the same subjects and genuinely
+ represent the specified perturbation. Without them, stability is explicitly
+ `not_evaluated`; request/scopingly plan resampling if the user requires it.
+
+ ```bash
+ python -m models.common.research_outputs run_models_output/subtyping_report/run_manifest.json \
+ --require subtype_assignments subtype_profiles subtype_counts subtype_figure report
+ ```
+
+ Use a new output directory for every run. With requested stability, add
+ `--require stability` to the final check. An absent requested output is incomplete
+ work, not an optional artifact silently omitted after training.
+
---
## Input / Output Summary
| Item | Format |
|---|---|
| Input | CSV; one row per subject |
| Required | numeric feature columns |
| Optional | configurable subject ID column |
| Assignments | `predictions.csv` with subject and subtype |
| Embedding | latent dimensions in `predictions.csv` |
| Metrics | `metrics.json` including silhouette and cluster count |
| Model | `checkpoint.joblib` |
| Provenance | `config.json`, `run_manifest.json` |
---
## Testing
```bash
pytest models/tests/test_extended_models.py -q
python skills/subject-subtyping/scripts/train_reference.py --help
+ pytest models/tests/test_research_output_skills.py -q
```
---
## Directory Reference
```text
models/subtyping/
├── estimators.py clustering and embedding implementations
└── train.py artifact-producing CLI
skills/subject-subtyping/
├── SKILL.md
- └── scripts/train_reference.py
+ └── scripts/
+ ├── train_reference.py
+ └── report_subtypes.py
```
---
## Reference
- Consensus clustering uses bootstrap co-assignment frequencies.
- NMF inputs are transformed to a non-negative scale before decomposition.
- PCA and autoencoder modes cluster the learned embedding rather than raw data.
---
Created At: 2026-07-26 HKT
- Last Updated At: 2026-07-29 HKT
+ Last Updated At: 2026-09-14 15:05:31.978 HKT
Author: chengwang96