git:20260916.63afbf9 to git:20260916.92a5830

- ---

- name: tlamatini-self-modify-inclusion

- description: Sweep the whole codebase and keep the SELF-MODIFY source snapshot complete — so the TlamatiniSourceCode/ tree that a `--self-modify` build ships carries every source/build asset Tlamatini needs to read, modify, and REBUILD herself, with heavy media/secrets correctly omitted+redacted and omitted binaries listed in the restore manifest. Invoke whenever you add/rename a new agent, source file, top-level dir, dependency, build input, prompt/identity file, migration, static/template asset, or any rebuild input — and ALWAYS after a feature like Blenderer or the self-update capability lands. Owns copy_source_assets.py and audits it against five invariants with a runnable sweep that GENERATES a snapshot and proves a rebuild's inputs all survive. Sibling of tlamatini-self-update-inclusion (which guards the runnable release).

- ---

- <!--

- ═══════════════════════════════════════════════════════════════════

- ✦ T L A M A T I N I ✦ — "one who knows"

- Created by Angela López Mendoza · @angelahack1

- Developer · Architect · Creator of Tlamatini

- Tlamatini Author Banner — do not remove (Angela's name is kept in every build)

- ═══════════════════════════════════════════════════════════════════

- -->

-

- # Tlamatini — Self-Modify Inclusion Sweep

-

- > **Audience:** Claude Code working ON the Tlamatini codebase for **Angela**.

- > **Goal:** guarantee the **source snapshot** a self-able-modify build ships

- > (`TlamatiniSourceCode/`, generated by `copy_source_assets.py`) carries **everything a

- > rebuild needs** — so a running Tlamatini can read → modify → **regenerate her own

- > `Tlamatini.exe`** without missing the minimal thing — while staying lean (media/secrets

- > out) and safe (no leaked keys).

-

- This is the twin of `tlamatini-self-update-inclusion`. That one guards the **runnable

- release** (what users download). **This one guards the SOURCE tree** Tlamatini carries

- of herself. They overlap on one fact: a rebuild from the snapshot must be able to re-run

- the *same* `build.py` — so **every input `build.py` consumes must survive the snapshot**.

-

- ### The acid test (the whole job in one sentence)

- > Can `cd <install>/TlamatiniSourceCode && python build.py --self-modify` succeed using

- > **only** the snapshot plus the documented restore steps? If yes, the snapshot is complete.

-

- ---

-

- ## The file this skill owns

-

- **`copy_source_assets.py`** (repo root) — generates the snapshot. Six knobs decide what

- ships:

-

- | Knob | Controls |

- |---|---|

- | `EXCLUDED_DIR_NAMES` | directory names pruned everywhere (`.git`, `node_modules`, `build`, `dist`, `pools`, `staticfiles`, `Temp`, `Templates`, `agents_backup`, `updater`, `TlamatiniSourceCode`, …) |

- | `KEEP_DIR_PATHS` | exact source-directory exceptions, including PDF.js's shipped `vendor/pdfjs/build/` API and worker; other build outputs stay excluded |

- | `EXCLUDED_EXTENSIONS` | file types dropped (`.pdf/.pptx/.png/.mp4/...`, binaries, generated state). **`.ico`/`.wav`/`.svg` are deliberately KEPT** (build-required) |

- | `EXCLUDED_FILE_NAMES` | exact files dropped (`db.sqlite3`, `data.keys`, `settings.local.json`, `_version.py`, …) |

- | `EXCLUDED_FILE_GLOBS` | patterns dropped (`*.version.txt`, `*_log.*`) |

- | `RESTORE_FROM_INSTALL` | binaries the build NEEDS but the snapshot OMITS → restored from the install at rebuild (`jd-cli.jar`, `XAIHT-Tlamatini.mp4`). The rebuild's escape hatch. |

- | `REQUIRED_SNAPSHOT_FILES` + `_wants_redaction()` | the completeness floor (raises if a critical file is missing) and the secret-redaction rules (config.json + agent config.yaml) |

-

- It also writes `_SOURCE_SNAPSHOT_MANIFEST.json` (counts/restore list) and

- `_REBUILD_INSTRUCTIONS.md` (the restore + rebuild runbook). The walk is **generic** — any

- new `.py/.js/.css/.yaml/.pmt/...` ships automatically. The danger is the *exclusions*

- wrongly dropping a rebuild input, an omitted binary missing from the restore manifest, a

- new always-present capability file not in `REQUIRED_SNAPSHOT_FILES`, or a secret leaking.

-

- ---

-

- ## The FIVE invariants

-

- ### Invariant 1 — INCLUDE: every rebuild SOURCE input survives

- Every file `build.py` reads to build (the build scripts, `requirements.txt`, all

- `--add-data` source trees, `optional/required_file_copies`, `optional_dir_copies` sources,

- `support_files`, the icon/wav/svg, every agent template, every `SKILL.md`, `prompt.pmt`,

- `Tlamatini.md`, `agentic_skill.md`, `monitoring-prompt.pmt`) must appear in the snapshot —

- **or** be in `RESTORE_FROM_INSTALL`. A source file dropped by an exclusion rule = a rebuild

- that fails or degrades.

-

- ### Invariant 2 — OMIT-LEAN, but RESTORE what the build needs

- Heavy media, dev trees, regenerable state, and secrets stay out (keep it ~10 MB, leak-free).

- But **anything omitted that the build genuinely consumes must be in `RESTORE_FROM_INSTALL`**

- (restored from the install) or covered by a regeneration step in `_REBUILD_INSTRUCTIONS.md`

- (e.g. `collectstatic` for `staticfiles`). Optional inputs (the `images` gallery) may

- degrade gracefully — decide per asset whether degradation is acceptable or it needs a

- restore entry.

-

- ### Invariant 3 — REDACT: no live secret survives

- `config.json` and every agent `config.yaml` are scrubbed to `<KEY goes here>` placeholders;

- `data.keys` is excluded outright. The snapshot must contain **zero** live keys.

-

- ### Invariant 4 — COMPLETENESS: critical files are guaranteed, not incidental

- `REQUIRED_SNAPSHOT_FILES` raises if a build-/run-critical source is missing. When you add a

- **new always-present capability** (a new build script, a new identity file, a flagship new

- agent), add its canonical file here so a future careless exclusion can't silently drop it.

-

- ### Invariant 5 — REBUILD COHERENCE: the manifest + instructions are true

- `_SOURCE_SNAPSHOT_MANIFEST.json` and `_REBUILD_INSTRUCTIONS.md` exist, `errors == []`, and

- the restore steps name real omitted binaries with correct install-relative sources.

-

- ---

-

- ## THE SWEEP — run this every time

-

- ### Step 0 — run the deterministic checker (does the heavy lifting)

-

- ```bash

- python .claude/skills/tlamatini-self-modify-inclusion/scripts/sweep_self_modify.py

- # add --keep to leave the generated snapshot under Temp/_self_modify_sweep for inspection

- ```

-

- It **actually generates** a snapshot into `Temp/` and verifies: it builds error-free with

- all `REQUIRED_SNAPSHOT_FILES`; **every resolvable `build.py` input is carried or restorable**

- (distinguishing source-drop FINDINGS from intentionally-omitted media/regenerable NOTES);

- no live secret survives (re-running copy_source_assets' own redaction logic on the OUTPUT +

- a config-file value scan); the restore manifest + instructions exist and name real binaries;

- and no source-like extension is in the exclude set. Exit code = findings (0 == clean). Fix

- every `[FINDING]`; read every `[note]` and confirm the degradation/regeneration is intended.

-

- ### Step 1 — diff since the last release and classify new paths

-

- ```bash

- git diff --name-status "$(git describe --tags --abbrev=0 --match 'v[0-9]*')"..HEAD

- # brand-new TOP-LEVEL repo entries — the highest risk for a wrongly-excluded rebuild input

- git diff --name-status "$(git describe --tags --abbrev=0 --match 'v[0-9]*')"..HEAD \

- | awk '$1=="A"{print $2}' | awk -F/ '{print $1}' | sort -u

- ```

-

- For each new path ask the taxonomy question below. New `.py/.js/.css/.yaml/.pmt/SKILL.md`

- inside already-walked trees auto-ship (note, no action). The risks are: a new **binary**

- build input, a new **excluded** dir/ext/name that hides source, a new **secret-bearing

- config file**, or a new **always-present** file that belongs in `REQUIRED_SNAPSHOT_FILES`.

-

- ### Step 2 — the judgment passes the script can't fully make

-

- ```bash

- # (a) New binary build inputs the build reads (icons/fonts/wav/jar/etc.) that EXCLUDED_EXTENSIONS would drop

- grep -nE "--add-data=|--icon|optional_dir_copies|required_file_copies|support_files" build.py | grep -iE "\.(ico|wav|svg|png|jpg|ttf|otf|jar|dll|bin|dat)"

- # (b) New config files that may carry secrets but AREN'T covered by _wants_redaction()

- grep -rilE "api[_-]?key|token|secret|password|client_secret" Tlamatini --include=*.json --include=*.yaml --include=*.yml \

- | grep -viE "/agents/.*/config.yaml$|agent/config.json$"

- # (c) New dependency / runtime added (cross-check the self-update skill too)

- git diff "$(git describe --tags --abbrev=0)"..HEAD -- requirements.txt

- ```

-

- - (a) Any binary the build needs and the snapshot drops → add to `RESTORE_FROM_INSTALL`

- (omit + restore) or to the KEEP exceptions (if small + truly source-required, like the

- `.ico`/`.wav`/`.svg` already are).

- - (b) Any NEW config file that can hold a live key → extend `_wants_redaction()` to scrub it

- (and confirm Step 0's redaction check then covers it), or exclude it by name.

- Treat `external_mcps.json` as a special preserved/tracked config: the snapshot and public

- build receive a scrubbed/default-only form, never the keyed developer catalog. Verify

- frozen-install contact book, or live External-MCP secrets. Verify current 88/66/108/29/197

- counts from the generated snapshot rather than copying prose counts.

-

- ---

-

- ## Asset taxonomy — type → ship in snapshot? → action

-

- | New asset | In snapshot by default? | Action |

- |---|---|---|

- | New `.py/.js/.css/.html/.yaml/.pmt/.proto/SKILL.md` (source) | ✅ auto (generic walk) | none |

- | New **agent** (`agent/agents/<x>/` source + config.yaml) | ✅ auto; config.yaml auto-**redacted** | none (verify redaction in Step 0) |

- | New **migration** (`.py`) | ✅ auto | none |

- | New **build script / build input file** at repo root | ✅ if not excluded | add to `REQUIRED_SNAPSHOT_FILES` if always-present |

- | New **small build-required binary** (`.ico/.wav/.svg`-class) | ✅ (those exts KEPT) | if a NEW binary ext, KEEP it (remove from excludes) or restore-map it |

- | New **heavy binary** the build needs (`.jar/.mp4/large media`) | ❌ omitted | add to `RESTORE_FROM_INSTALL` (+ it lands in `_REBUILD_INSTRUCTIONS.md`) |

- | New **optional media** (gallery image) | ❌ omitted | leave omitted if degradation OK; else restore-map |

- | New **config file with secrets** | ✅ shipped — ⚠️ maybe UNREDACTED | extend `_wants_redaction()` or exclude by name |

- | New **runtime-state / generated dir** | should be ❌ | add its name to `EXCLUDED_DIR_NAMES` |

- | New **runtime / evidence dir INSIDE a shipped tree** (e.g. `security/security_logs/`) | ❌ must be **EXCLUDED** | add the dir name to `EXCLUDED_DIRS` in `copy_source_assets.py`. It is state, not source — and here it is the operator's forensic evidence, which can contain their own usernames, IPs and command lines. It must never travel in a snapshot |

-

- The single most dangerous omissions: a **source file hidden by an over-broad exclusion**

- (Step 0 flags it as a source-drop FINDING) and a **new secret-bearing config that

- `_wants_redaction()` doesn't cover** (Step 2b — the raw scan in Step 0 catches obvious key

- shapes, but a novel config path needs the redaction rule extended).

-

- ---

-

- ## Where to make each fix (`copy_source_assets.py`)

-

- - **A source file got excluded** → narrow the offending rule in `EXCLUDED_EXTENSIONS` /

- `EXCLUDED_FILE_NAMES` / `EXCLUDED_FILE_GLOBS` / `EXCLUDED_DIR_NAMES`, or add a KEEP carve-out.

- - **An omitted binary the build needs** → add `"snapshot/rel/path": "install/rel/source"` to

- `RESTORE_FROM_INSTALL` (it auto-renders into `_REBUILD_INSTRUCTIONS.md`).

- - **Guarantee a critical file** → add its snapshot-relative path to `REQUIRED_SNAPSHOT_FILES`.

- - **A new secret-bearing config** → extend `_wants_redaction()` to return `'json'`/`'yaml'`

- for it (and add the key suffix to `_SECRET_KEY_RE` if it's a novel secret-key name).

- - **A new state/output dir leaking in** → add its name to `EXCLUDED_DIR_NAMES`.

-

- ---

-

- ## Done criteria (all must hold)

-

- 1. `sweep_self_modify.py` exits clean (no `[FINDING]`); every `[note]` is reviewed + intended.

- 2. Every new top-level repo path from the since-last-tag diff is classified and handled.

- 3. The generated snapshot builds error-free and contains every `REQUIRED_SNAPSHOT_FILES` entry.

- 4. Every `build.py` SOURCE input is in the snapshot or in `RESTORE_FROM_INSTALL`.

- 5. Zero live secrets in the snapshot (redaction check + value scan pass).

- 6. The manifest + `_REBUILD_INSTRUCTIONS.md` are accurate; restore targets are real.

- 7. `python -m ruff check copy_source_assets.py` is clean.

- 8. (Ideal) you eyeballed a `--keep` snapshot and spot-confirmed the newest feature's source

- files are physically present.

-

- ---

-

- ## Companion references

- - `tlamatini-self-update-inclusion` — the sibling skill guarding the **runnable release**

- (`build.py` carriers + the update swap). Run BOTH after any feature: one proves users can

- *receive* the feature, this one proves Tlamatini can *rebuild herself with* it.

- - `docs/claude/architecture.md` → *Self-Knowledge & Self-Modification* and

- `docs/claude/gotchas.md` → *Building & Packaging* for the `--self-modify` flag mechanics.

- - `VERSIONING.md` — the rebuild produces the next version from the git tag.