Immutable. This exact content is served forever at /api/v1/blob/2bbbc4d5d0508df8.
---
name: memory-management
plugin: memory-management
description: "Tiered memory system for cognitive continuity across agent sessions. Manages hot cache (session context loaded at boot) and deep storage (loaded on demand). Use when: (1) starting a session and loading context, (2) deciding what to remember vs forget, (3) promoting/demoting knowledge between tiers, (4) user says 'remember this' or asks about project history."
allowed-tools: Read, Write
---
# Memory Management
Tiered memory system that makes an AI agent a continuous collaborator across sessions.
## Architecture
The memory system has six tiers, configurable per project:
```
Tier 1: HOT CACHE (always loaded at boot -- ~200 lines target)
+-- <primer_file> Role, identity, constraints
+-- <boot_digest_file> Tactical status, active tasks
+-- <boot_contract_file> Immutable constraints
+-- <snapshot_file> Cognitive Hologram (1 sentence per file)
Tier 2: RLM SUMMARY LEDGER (fast keyword lookup -- loaded on demand)
+-- <summary_cache_file> Pre-generated text summaries: docs, protocols, research
Plugin: rlm-factory | Skill: rlm-search | Profile: project
+-- <tool_cache_file> Pre-generated text summaries: plugins, skills, scripts
Plugin: rlm-factory | Skill: rlm-search | Profile: tools
Tier 3: VECTOR STORE (semantic embedding search -- loaded on demand)
+-- <vector_db_backend> ChromaDB via vector-db plugin / vector-db-agent skill
Profile: knowledge | Port: configured in vector_profiles.json
Tier 4: DEEP STORAGE (filesystem -- authoritative source, loaded on demand)
+-- <domain_data_dir>/ Research topics: {topic}/analysis.md
+-- <design_docs_dir>/ ADRs, RFCs
+-- <governance_dir>/ Protocols, playbooks
Tier 5: VAULT (Obsidian -- linked knowledge graph, loaded on demand)
+-- <vault_dir>/ Plugin: obsidian-wiki-engine
Skills: obsidian-vault-crud, obsidian-canvas-architect,
obsidian-graph-traversal, obsidian-bases-manager,
obsidian-wiki-builder, obsidian-query-agent,
obsidian-rlm-distiller, obsidian-wiki-linter
Env: VAULT_PATH or OBSIDIAN_VAULT_PATH
Tier 6: SOUL (external persistence -- optional, synced at session seal)
+-- <traces_file> Plugin: project-specific (e.g. huggingface-utils)
e.g. lineage/, data/, soul_traces.jsonl on HF Hub
```
Projects define their own file paths for each slot. Tiers may be omitted or added based on project complexity.
## Lookup Flow (3-Phase Search Protocol)
When searching for information, ALWAYS escalate in order. Never skip ahead.
```
Query arrives ->
1. HOT CACHE Instant. Boot files cover ~90% of context needs.
2. DEEP STORAGE (topic/decision) Load specific domain dir or design doc by subject.
3. RLM SUMMARY LEDGER (Phase 1) Keyword search via rlm-factory:rlm-search skill.
4. VECTOR STORE (Phase 2) Semantic search via vector-db:vector-db-search skill.
5. GREP / EXACT SEARCH (Phase 3) rg/grep scoped to paths from Steps 3 or 4.
6. Ask user Unknown? Learn it and persist it.
```
### Phase 1 -- RLM Summary Scan (Table of Contents)
RLM is **amortized prework**: each file read ONCE, summarized ONCE, cached as plain text JSON.
Searching summaries is O(1) keyword lookup -- no embeddings, no inference.
Trigger the `rlm-factory:rlm-search` skill, providing the profile and search term.
**Use Phase 1 when:** You need to understand what a file does, find which file owns a feature,
or navigate the codebase without reading individual files.
**Escalate to Phase 2 when:** The summary is insufficient or no match found.
### Phase 2 -- Vector Store Semantic Search (Back-of-Book Index)
Embedding-based nearest-neighbor search across all indexed chunks. Returns ranked
parent chunks with RLM Super-RAG context pre-injected.
Trigger the `vector-db:vector-db-search` skill, providing the query, profile, and limit.
**Use Phase 2 when:** You need specific code snippets, patterns, or implementations.
**Escalate to Phase 3 when:** You have a file path (from Phase 1 or 2) and need an exact line.
### Phase 3 -- Grep / Exact Search (Ctrl+F)
Precise keyword or regex match. Always scope to paths discovered in earlier phases.
```bash
# Scoped to a specific path (use paths from Phase 1/2)
grep_search "VectorDBOperations" ../../skills/
# Ripgrep for regex
rg "def query" ../../ --type py
```
**Anti-patterns:** Never run a full-repo grep without scoping. Never skip Phase 1.
## Dependencies
### `rlm-factory` -- RLM Summary Ledger (Tier 2)
| Component | Value |
|:----------|:------|
| **Plugin** | `.agents/skills/` (rlm-curator, rlm-search, rlm-init, rlm-distill-agent) |
| **Skill (write)** | `skills/rlm-curator/` -- distill, inject, audit, cleanup |
| **Skill (read)** | `skills/rlm-search/` -- query the ledger |
| **Skill (Phase 1 search)** | `rlm-factory:rlm-search` |
| **Skill (write/inject)** | `rlm-factory:rlm-curator` |
| **Skill (audit coverage)** | `rlm-factory:rlm-curator` |
| **Skill (shared config)** | `rlm-factory:rlm-curator` |
| **Cache files** | `.agent/learning/rlm_summary_cache.json` (docs), `.agent/learning/rlm_tool_cache.json` (tools) |
### `vector-db` -- Vector Store (Tier 3)
| Component | Value |
|:----------|:------|
| **Plugin** | `.agents/skills/` (vector-db-search, vector-db-ingest, vector-db-launch, vector-db-init) |
| **Skill** | `skills/vector-db-agent/` -- ingest, query, operations |
| **Skill (Phase 2 search)** | `vector-db:vector-db-search` |
| **Skill (ingest files)** | `vector-db:vector-db-ingest` |
| **Skill (operations)** | `vector-db:vector-db-search` |
| **Skill (config)** | `vector-db:vector-db-search` |
| **Backend** | ChromaDB (`chromadb.HttpClient` with `PersistentClient` fallback) |
### `obsidian-wiki-engine` -- Linked Vault + LLM Wiki (Tier 5)
| Component | Value |
|:----------|:------|
| **Plugin** | `.agents/skills/` (obsidian-vault-crud, obsidian-init, obsidian-wiki-builder, obsidian-query-agent, obsidian-rlm-distiller, obsidian-wiki-linter, obsidian-canvas-architect, obsidian-graph-traversal) |
| **Skill: vault setup** | `obsidian-wiki-engine:obsidian-init` -- prerequisites, `.obsidian/` config, exclusion filters |
| **Skill: read/write notes** | `obsidian-wiki-engine:obsidian-vault-crud` -- atomic create/read/update/append |
| **Skill: build wiki nodes** | `obsidian-wiki-engine:obsidian-wiki-builder` -- Karpathy-style concept nodes |
| **Skill: query wiki** | `obsidian-wiki-engine:obsidian-query-agent` -- 3-phase RLM+vector+grep search |
| **Skill: distill RLM** | `obsidian-wiki-engine:obsidian-rlm-distiller` -- generates RLM summary layers |
| **Skill: semantic lint** | `obsidian-wiki-engine:obsidian-wiki-linter` -- inconsistencies, gaps, stale articles |
| **Config** | `.agent/learning/rlm_wiki_raw_sources_manifest.json` (sources), `.agent/learning/vector_profiles.json` (wiki profile) |
| **Requires** | `pip:ruamel.yaml` (lossless YAML frontmatter), Obsidian Desktop |
| **Env** | `VAULT_PATH` -- absolute path to the vault root |
## Promotion / Demotion Rules
### Promote to Hot Cache when:
- Knowledge is referenced in 3+ consecutive sessions
- It's critical for active work (current spec, active protocol)
- It's a constraint or identity anchor
### Demote to Deep Storage when:
- Spec/feature is completed and merged
- Governing document is superseded by newer version
- Topic research is concluded
- Technical decision is ratified (move from draft to archive)
### What Goes Where
| Type | Hot Cache | On-Demand Tier |
|------|-----------|----------------|
| Active tasks | Boot digest | -- |
| Identity/role | Primer file | -- |
| Constraints | Boot contract | -- |
| Session state | Snapshot file | Tier 6 Soul (traces) |
| Research topics | Summary in snapshot | Tier 4: `domain_data_dir/{name}/` |
| Design decisions | Referenced by ID | Tier 4: `design_docs_dir/{id}_{name}.md` |
| Governing docs | Referenced by ID | Tier 4: `governance_dir/{id}_{name}.md` |
| Plugins/scripts/tools | -- | Tier 2: RLM Summary Ledger (tool cache) |
| Docs/protocols/research | -- | Tier 2: RLM Summary Ledger (summary cache) |
| System docs | -- | Tier 2 RLM + Tier 3 Vector Store |
| Linked notes, canvases | -- | Tier 5: Vault (Obsidian) |
| External persistence | -- | Tier 6: Soul (HuggingFace or equivalent) |
## Session Memory Workflow
### At Session Start (Boot)
1. Load hot cache files in order (primer -> contract -> digest -> snapshot)
2. Integrity check validates snapshot is current
3. If snapshot stale -> flag for refresh at session end
### During Session
- **New learning** -> Write to `<domain_data_dir>/{topic}/`
- **New decision** -> Create design document draft
- **New tool** -> Register in tool inventory
- **Correction** -> Update relevant file + note in disputes log if contradicting
### At Session End (Seal)
1. Update snapshot file with new content learned this session
2. Seal validates no drift since last audit
3. Persist traces to external storage (if configured)
## Conventions
- **Hot cache target**: ~200 lines total across all boot files
- **Snapshot**: 1 sentence per file, machine-readable
- **Topic folders**: `lowercase-hyphens/`
- **Document numbering**: 3-digit, sequential
- **Always capture** corrections and contradictions in a disputes log
## Configuration
Projects configure the memory system by setting file paths in their project-specific plugin:
| Variable | Purpose |
|:---------|:--------|
| `MEMORY_PRIMER_FILE` | Path to cognitive primer / role definition |
| `MEMORY_BOOT_DIGEST` | Path to tactical boot digest |
| `MEMORY_BOOT_CONTRACT` | Path to immutable constraints |
| `MEMORY_SNAPSHOT_FILE` | Path to learning snapshot (hologram) |
| `MEMORY_DOMAIN_DIR` | Directory for domain research |
| `MEMORY_DESIGN_DIR` | Directory for design docs (e.g. ADRs) |
| `MEMORY_GOVERNANCE_DIR` | Directory for governing docs (e.g. Protocols) |
## Architecture Diagrams
| Diagram | What It Shows |
|:--------|:--------------|
| [memory_architecture.mmd](assets/resources/memory_architecture.mmd) | Full 4-tier memory system with exact plugin/skill/script names per tier |
| [memory_lookup_flow.mmd](assets/resources/memory_lookup_flow.mmd) | 3-phase search sequence: Hot Cache -> RLM Ledger -> Vector Store -> Grep |
| [memory_session_lifecycle.mmd](assets/resources/memory_session_lifecycle.mmd) | Session Boot -> Active -> Seal lifecycle with all event types |