git:20260925.99c7a9e to git:20260925.4486774

---

name: song-analysis

description: Use when analyzing a song recording — extracting tempo/downbeats/key, separating stems, transcribing bass to MIDI, identifying chords per bar, timing lyrics, reading drum patterns, or producing a chord+lyric chart for a band. Covers madmom, demucs (htdemucs_ft), CREPE, basic-pitch, chroma chord proposal, and Whisper word timing.

---

# Song Analysis — recording → bars, chords, lyrics, chart

Song-agnostic pipeline, validated over many real projects (dream pop, Greek

laiko, punk reinterpretations). Works for any song, genre, or key.

**References (read on demand):**

- `references/environment-setup.md` — first-time venv install + per-song folder layout

- `references/chart-and-lyrics.md` — chart structure, the 6 lyric-placement rules, HTML output

- `references/drum-pattern-analysis.md` — kick/snare pattern reading from the drums stem

- `references/modal-theory.md` — power-chord shorthand, mode tables, Byzantine ≠ Phrygian

## Inputs

1. **Audio file** — mp3/wav of the song (full mix).

2. **Lyrics** — canonical text from an official source. Section labels (Verse, Chorus,

Bridge…) come from this, not from the analyzer.

## Tool per step

| Step | Tool | Why |

|---|---|---|

| Pulse (beats only) | **madmom** `DBNBeatTrackingProcessor` | No meter assumed — see Phase 1a |

| Meter / bar length | **`scripts/detect_meter.py`** | Sweeps every cycle 2–25; finds odd meters |

| Key (melodic mode) | **librosa** chroma + Krumhansl | Signal only — see Trap 2 |

| Stem separation | **demucs `htdemucs_ft`** (4-stem) | Cleanest bass; NEVER `htdemucs_6s` (bleeds piano/guitar into bass) |

| Polyphonic parts → MIDI | **basic-pitch** | Piano/guitar/pads, double-stops |

| Lyrics + word timing | **mlx-whisper `large-v3-turbo`** (Apple Silicon) | Word timestamps, hallucination-filterable |

| Drum-pattern read | **librosa** band-limited onsets | see reference |

| Chart output | Python → self-contained **HTML** | see reference |

## Phase 1 — Foundation (pulse + meter + key)

### 1a. Track the pulse WITHOUT assuming a meter

```python

from madmom.features.beats import RNNBeatProcessor, DBNBeatTrackingProcessor

import librosa, numpy as np

act = RNNBeatProcessor()(audio_path)

beat_times = DBNBeatTrackingProcessor(fps=100)(act) # beats only, NO bar assumption

beat = float(np.median(np.diff(beat_times)))

pulse_bpm = 60.0 / beat

# Key via librosa chroma_cqt mean + Krumhansl correlation → melodic mode ONLY (Trap 2)

```

**Never pass `beats_per_bar=(3,4)`.** A constrained candidate list cannot return "none of

these" — it returns the least-bad of the options you supplied, with full confidence, and

every downstream number (bar counts, per-bar chord cells, drum histograms) silently

inherits the error. Get the pulse first; derive the meter in 1b.

### 1b. Derive the meter by sweeping every cycle length

```bash

--bass stems/<song>/bass.wav --other stems/<song>/other.wav \

```

**Always pass `--bass` and `--other` too.** Plenty of songs have no drum kit, or a

percussion part that is an undifferentiated pulse carrying no accent — and the cycle is

still there, articulated by the bass and the harmony instead. The signature is a drum

contrast stuck near 1.2 while the bass reaches 3–4 and resolves the cycle cleanly. The

Phase-folds band-limited kick / snare / hat onsets onto every cycle length from 2 to 25

pulses and ranks by **accent contrast** = loudest position in the cycle ÷ quietest.

Run it AFTER Phase 2 so it has a drums stem; a full mix works but reads muddier.

Reading the output:

| Output | Means | Do |

|---|---|---|

| Consensus, confidence HIGH | Real cycle, clear of unrelated periods | Use it. Odd winner → odd meter |

| Winner is prime (7, 11, 13) | Cannot be a phrase of anything smaller | Almost certainly the meter |

| Winner = 2× a strong period | The half is the bar, the double is two bars | Take the fundamental |

**The sweep can only find cycles on the pulse grid you hand it.** If the tempogram peaks

at twice the tracked pulse, the beat tracker is reading half-time: re-run on the doubled

grid, because a cycle of 8 there shows up as 4 here. Check the tempo octave *before*

trusting any cycle length.

**A song with no percussion at all cannot be metered this way** — say so and move on

rather than straining the harmonic stem for an answer it does not contain.

**The two strongest positions in an odd cycle mark its internal split.** An 11 with peaks

at 1 and 7 is 6+5; peaks at 1 and 5 would be 4+7.

Save as `analysis/foundation.json`: `pulse_bpm`, `beat_times`, `beats_per_bar` (from 1b,

**Sanity checks:** BPM stable (no jumps); watch half/double-time on slow songs (60–90

BPM) — count along the audio; cross-check `num_bars` against

`duration ÷ (beats_per_bar × beat)`. A large mismatch means rubato, a wrong tempo octave,

or a wrong meter.

## Phase 2 — Stems

```bash

```

on a previous song.

**Fast offline fallback (Apple Silicon):** plain htdemucs via MLX, weights already local —

28 s for a 4-minute track (8.7× realtime):

```bash

<audio-analysis>/mlx-demucs/.venv/bin/mlx-demucs <song.mp3> -o stems_mlx -v

```

Call the venv binary **directly**. `uv run mlx-demucs` re-resolves dependencies over the

network and hangs 10+ min even though the venv is already complete. The MLX port is within

0.03% of the PyTorch htdemucs reference, so it's a sound basis for chords and bass; re-run

on `_ft` later only if quality looks marginal. Note its README lists `htdemucs_ft` as a

supported model, but only plain htdemucs weights ship in `weights/`.

**Quality check:** listen to `bass.wav` alone — piano/guitar bleed means separation

struggled; try plain `htdemucs` as fallback.

## Phase 3 — Bass → MIDI

cleanup rules, use the `bass-transcribe` skill — same pipeline. Use **basic-pitch** only

**Spell pitch classes from the key signature, not from a fixed table.** Flat spelling

(C, D♭, D, E♭ … B♭, B) is the right default for flat and neutral keys — it matches how

guitarists read pop/indie charts. But applying it blindly in a **sharp key produces

nonsense**: in F♯ minor (three sharps) the flat table renders the tonic as `G♭m`

instead of `F♯m`, and the dominant as `D♭7` instead of `C♯7`. Rule: if the

key signature has sharps, use sharp spelling (C♯, D♯, F♯, G♯, A♯); if flats or none, use

flats.

**Pitfalls:**

- Sub-0.5 s detections are noisy; ≥0.7 s sustained notes are usually right.

- Bass doesn't always play roots — it pedals, walks, sits on 3rds/5ths. Bass note is a

*clue*; annotate in chart only when bass ≠ chord root.

- **Bass enters late** in most songs. Find the bass-entry bar (first clearly audible bar)

and suppress all bass-derived annotations before it — pre-entry content is stem bleed.

## Phase 4 — Lyric timing (Whisper on the VOCALS STEM)

Whisper is far more accurate on isolated vocals than the full mix.

Map each word's start time to (bar, beat):

```python

for i in range(len(downbeats)-1):

if downbeats[i] <= t < downbeats[i+1]:

return None

```

## Phase 5 — Sections (manual, from lyrics + audio)

For each canonical lyric line, find its first bar via Whisper + `locate()`. Section

boundaries = where each labeled section's first line lands. **Don't trust automatic

section detection** (librosa recurrence segmentation is ±1 bar off around bridges/outros);

the lyrics + word timing are far more reliable.

1. **Chroma**: average `librosa.feature.chroma_cqt(other.wav)` over the cell's frames.

2. **Score all 24 triad templates** (12 maj + 12 min, binary root/3rd/5th, sum-normalized)

by dot product with the cell chroma.

3. **Bass-root constraint** (only when `bar >= BASS_ENTRY_BAR`): if one bass pitch class

has ≥0.5 s sustain in the cell, restrict candidates to the maj+min triads on that root —

| `MAJOR_BIAS_MARGIN` | root+quality |

|---|---|

| 0.02 | 54.9% |

Per-song tuning: `BASS_ENTRY_BAR`, template set.

## Phases 6–9 — Chart

Read `references/chart-and-lyrics.md` before building the chart. Core invariants:

- One row per section; parallel sections get identical row splits.

- **Lyric phrases anchor at the chord they resolve INTO** (Rule 1 — the big one).

- Output: single self-contained HTML, print-friendly, harmonic-notes block at bottom.

## Hard rules — the cardinal traps

Bass movement alone is a suspicion signal, never the decision.

### Trap 2 — "Minor key" ≠ minor chords

Key estimators report the **melody's mode** (the PCP is melody-dominant), not chord

quality. Many songs put **major triads under a minor melody** — that's the bittersweet

sound. Default each chord to major, then verify: play root+maj3+5 against the melody; if

the maj-3rd fights the melody's ♭3, go minor. Never propagate the key label to chord

quality. Also: many modal songs pivot between relative major and minor — label the key as

a pair (`E♭ / Cm`) when both centers appear.

### Trap 3 — The meter you never tested for looks like no meter at all

Folding onsets onto the wrong period does not produce a *low* score, it produces a

**flat** one. An 11-beat cycle folded onto 2, 3, 4 or 6 smears to near-perfect uniformity,

because 11 shares no factor with any of them.

Calibration — read your own contrast numbers against this:

| Top contrast in the sweep | Reading |

|---|---|

| 1.0–1.3 at *every* period tested | Wrong periods tested, or genuinely no accent. Widen the sweep |

| 1.5–2.5 | Weak or ambiguous — report it as such |

| 3–10 at one period, ≤2.5 at every unrelated one | A real cycle. Trust it |

A coprime cycle read through the wrong fold lands squarely in the first row: uniform, not

low.

**Uniform contrast at every period you tested is not a finding. It means you have not

tested the right period yet.** Widen the sweep before concluding anything, and never

report "no time signature" on the strength of a narrow sweep.

Corollary: assume odd meters are live. 5/4, 7/8 and 11/8 are ordinary in Balkan,

Byzantine, prog and much devotional music — exactly the material this pipeline gets used

on. Coprime blindness makes them invisible to a 4/4-shaped test.

**Verify with a player.** All automated signals are wrong ~30% of the time on

modal-mixture songs. A musician who knows the song is ground truth — plan a verification

pass before declaring the chart done.

## Lessons learned

1. `htdemucs_ft`, never `htdemucs_6s` — 6-stem bleeds piano/guitar into bass MIDI.

2. Trust the user's ear over any analyzer disagreement.

3. Chart bar = audio bar.

4. Whisper timing is precise; musical placement is not strict timing (Rule 1).

6. Chord-recognition tools are starting points, never ground truth.

7. Each chart correction touches ~3 places (chord dict, row layout, cascading lyrics).

8. Mirror parallel sections visually.

9. Section-boundary pickups: overlay in the same cell, don't displace.

10. Never hand an analyzer a candidate list it can't say "none of these" to — a

constrained `beats_per_bar` and a key-gated `MAJOR_BIAS_MARGIN` both produced

confident wrong answers that survived every downstream check.

11. Ask the user to count. A player's count settles in seconds what a sweep can only

rank, and it outranks the sweep.

12. A static upper voicing over a moving bass reads as several chords to a triad matcher.

Before reporting a progression, pool chroma **per bar in the instrument's own

register** (high-pass away the bass) and check whether the set actually changes — a

drone with a walking bass is one chord, not four.

13. `basic-pitch` duplicates notes at exact octaves and invents low-register content that

the stem does not contain. Floor the output at the instrument's real range (C3 for a

piano out of `other.wav`) before pushing anything to Ableton.

## Local environment (this machine)

- Tooling root: `/Users/peripan/dev/abletonAI/audio-analysis/`

- `.venv-bp/bin/python` — madmom, librosa, basic-pitch, crepe, pretty_midi.

`.venv-demucs/bin/python` — demucs.

- **Whisper is NOT in any audio-analysis venv.** It lives at

`/Users/peripan/mlx-openai-whisper/bin/python`, and only

`mlx-community/whisper-large-v3-turbo` is cached — naming `whisper-large-v3-mlx`

triggers a ~3 GB download.

- Source docs (read-only): `WORKFLOW-song-analysis.md` in the tooling root.