update-culture git:20260826.de6d6bbA

Immutable. This exact content never changes and is served at /api/v1/blob/f6c4d0c9706ce15f.

---
name: update-culture
description: Refresh the Култура (culture) data behind /culture — the НФЦ (Национален филмов център) film-subsidy corpus in data/culture/films.json + overview.json, parsed from the per-year .xls registers of financed films/series at nfc.bg, plus НФК grant success rates, the artistic-commission compositions ("кой решава"), and the Sofia + читалища municipal streams. Also refreshes the МК ДКИ register (data/culture/dki_register.json) — the ministry's own listing of the държавни културни институти it is the principal of, with each institute's director and seat, reconciled against the four-list culture allowlist. Use when the daily watch report flags `nfc_film_register`, `ncf_grant_results`, `nfc_commissions`, or `mc_dki_register` as changed, when the user asks to refresh culture / кино / филмови субсидии / НФЦ / комисии data, or after a fresh git clone if data/culture/*.json is missing.
---

# update-culture

Refreshes the culture-spending data served at `/culture` (the dedicated dashboard)
and read by the `cultureOverview` / `topCultureGrantees` / `filmSubsidyForProducer`
AI tools. Phase 1 covers the **НФЦ film-subsidy register**; НФК grants, the pack on
`/awarder/000695160`, and the jury-transparency tile are later phases (see
`docs/plans/kultura-view-v1.md`).

## Source

The НФЦ **Единен публичен регистър** of financed films/series — one `.xls` per year
(2014–2025), all under `nfc.bg/wp-content/uploads/2022/07/…`. Filenames are NOT
uniform across years, so the explicit map lives in `scripts/culture/sources.ts`.
No WAF/login. Amounts are historical BGN → EUR at the fixed rate 1 EUR = 1.95583 BGN.

Two format families the parser handles (see `scripts/culture/ingest.ts`):
- **2022–2025**: `Вид · Наименование · Рег.№ · Продуцент · Субсидия лв · Бюджет лв ·
  Протокол на ФК`, with "Игрално кино:" section rows.
- **2014–2021**: `№ · Филм · Рег.№ · Продуцент · Държавно финансиране лв · Заповед`,
  multi-sheet, discipline embedded in the reg-number / title.

Discipline is classified from the **reg-number letter** (И=игрално, Д=документално,
А=анимационно) — reliable across both families — with a title-prefix fallback.

### The МК ДКИ register (`mc_dki_register`)

A SECOND, unrelated source: МК's own three listings of the **държавни културни
институти** it is the principal of — `направление Музика и танц` (opera,
philharmonic, symphonietta), `направление Театър` (drama + puppet) and
`направление Художествено образование` (the art schools). URLs in
`scripts/culture/dki/sources.ts`.

Three things about it decide how it may be used:

- **It carries NO ЕИК.** Every id in the artifact is resolved by NAME against
  `contracts` ∪ `tenders` (`scripts/culture/dki/resolve.ts`), which REFUSES an
  ambiguous name rather than grading a guess — two register names are too
  generic to identify and stay unresolved on purpose.
- **It is not the whole of МК.** ~103 second-level spending units, ~74 of them
  ДКИ; these three pages list 70, and the national museums, galleries and
  library are on no ДКИ page at all. The museums register at
  `/документи/регистри-1/` is deliberately NOT folded in, and the reason is
  sharper than „mostly municipal": its `Форма на собственост` column shows
  `държавен` **spans principals** — of the 17 state-owned museums, 5 of the 10
  that resolve to an EIK are МО or БАН. Folding it in wholesale would import
  those into a culture roll-up. It is a ~17-row hand-classified follow-up, not a
  permanent boundary.
- **It never becomes the allowlist.** `src/lib/kulturaReferenceData.ts` stays
  hand-classified by principal. This is the independent evidence it is checked
  against, and each source sees what the other structurally cannot: the corpus
  sweep is blind to a ДКИ that never ran a ЗОП procedure, and the register is
  blind to everything МК does not list.

mc.government.bg serves an **incomplete certificate chain**, so both the ingest
and the watcher pass `insecureTls` — curl and browsers accept it, Node's CA list
does not. Read-only public pages.

## Run

```bash
npx tsx scripts/culture/ingest.ts          # НФЦ films → films.json + overview.json (start local Postgres FIRST: npm run db:pg:up)
npx tsx scripts/culture/ingest.ts --force  # re-download every year's .xls
npx tsx scripts/culture/ncf_grants.ts      # НФК grant results → grants.json (needs pdftotext)
npx tsx scripts/culture/build_oblast.ts    # state institutes by oblast (needs Postgres)
npx tsx scripts/culture/write_commissions.ts # НФЦ artistic-commission compositions → commissions.json
npx tsx scripts/culture/sofia_program.ts   # Sofia Програма „Култура" + читалища → municipal.json (needs pdftotext)

npm run culture:dki                        # МК ДКИ register → dki_register.json (DRY RUN; needs Postgres)
npm run culture:dki -- --apply             # …and write it
npm run culture:dki -- --apply --offline   # re-parse the cached HTML, no fetch
npx vitest run scripts/culture/dki/        # the parser + reconciliation gates — ALWAYS after --apply

npx tsx scripts/culture/enrich_producers.ts # REPAIR ONLY — re-link top producers → EIK (needs Postgres)
```

**The producer→EIK links are part of the ingest, not a step you can forget.** The НФЦ
register has no EIK, so the `/company/:eik` link on every top-producer row comes from
`scripts/culture/producer_eik.ts`, which resolves a producer name to a company EIK ONLY
where it matches exactly one Commerce-Registry company (unique match); ambiguous names
("Клас", "АРС") and no-matches are left unlinked rather than guessed (plan §6). `ingest.ts`
calls it **before** it writes `overview.json`, so a refresh can never leave the file
EIK-less — which it did on 2026-07-31, when the linking was a separate script and every
producer link went dead with nothing failing.

That linking needs **local Postgres** (`tr_companies`), so bring it up before the ingest:

```bash
npm run db:pg:up
```

Without it the ingest still succeeds (the parse is Postgres-free) but prints a loud
`⚠ producer→EIK linking SKIPPED` and carries the previous `overview.json`'s EIKs forward.
`enrich_producers.ts` is then the repair: start Postgres and run it to re-link in place.
Its other use is after a TR reload, when the company corpus has moved but the films
have not.

The **НФК grants** ingest (`ncf_grants.ts`) parses the класиране PDFs listed in the
curated `NCF_RESULTS` map into `data/culture/grants.json` — the applied-vs-funded
success rate per art discipline. Results URLs are scattered across ncf.bg news posts
(no clean index), so add each new session's PDF URL to `NCF_RESULTS` before running.
Requires `pdftotext` (poppler). The **oblast** build (`build_oblast.ts`) needs Postgres
(`awarder_seats` + `contracts_list`) and is stable (the institute allowlist rarely
changes).

The **commission** compositions (`write_commissions.ts`) are HAND-KEYED from the НФЦ
executive-director appointment order ("Назначаване съставите на НХК…"), which changes
each ~6-month mandate. When the `nfc_commissions` watch flags a new order, download the
newest one from the nfc.bg „Заповеди" page, read it (`pdftotext -layout`), and update the
mandate window + 21 members in the generator. The **municipal** streams (`sofia_program.ts`)
parse the Столична програma „Култура" класиране PDF (dropped at `raw_data/culture/sofia_spk_<year>.pdf`)
and carry the hand-keyed читалища national figures.

Writes `data/culture/films.json` (per-film corpus) and `data/culture/overview.json`
(precomputed dashboard blob: totals, by-year, by-discipline, top producers,
top-10 concentration). The script **self-verifies** (plan §9): it asserts every
year parses > 0 rows and that the flat total reconciles to the per-year sum, and
refuses to write a partial artifact on failure.

## After a successful run

1. Eyeball the printed per-year counts + the Σ line (`~944 films · €94.9M · 324
   producers · top-10 22%` for the 2014–2025 baseline, after de-duping the 5
   identical rows the register ships).
2. Check the EIK line (`18/25 top producers linked to a unique EIK` — never a
   `⚠ … SKIPPED`) and confirm the link count did not fall. Compare COUNTS, not the
   diff: `topProducers` is sorted by `eur`, so any rank change rewrites those blocks
   and a `grep '^-.*"eik"'` over the diff reports losses that did not happen.
   ```bash
   echo "was $(git show HEAD:data/culture/overview.json | grep -c '"eik"') → now $(grep -c '"eik"' data/culture/overview.json)"
   ```
3. Commit `data/culture/*.json`, then publish:
   ```bash
   npm run bucket:sync:paths -- culture
   ```
   (`cp -Z` — GCS serves identity; avoid `gsutil -m` on macOS. The older form of
   this line read `bucket:sync data/culture/`, which is not a command: `bucket:sync`
   is the full-tree rsync and takes no path argument.)

   ⚠️ **`data/culture/derived/hub_stats.json` is NOT refreshed by this skill, and
   this step is not what publishes it.** It is written by `db:gen-culture-hub-stats`,
   the last step of `db:refresh`, from contracts / tenders / fund_projects /
   agri_subsidies / person_role / interreg_partners — none of which this skill
   touches. So the file moves when `update-procurement` runs, while the sync that
   would ship it lives here, behind a trigger (`nfc_film_register`,
   `ncf_grant_results`, `nfc_commissions`, `mc_dki_register`) that never fires for
   it. Measured 2026-08-21: the blob had returned **404 for two days** — committed,
   generator in the chain, never uploaded — and `/culture` served its tiles without
   numbers at a 200, because the hook degrades a 404 to „no figure" on purpose.
   The orchestrator now checks all six such artifacts unconditionally
   (`npm run db:check-generated`, process-watch-report step 8); the sync above
   still covers it whenever this skill runs, which is a second chance, not the
   mechanism.
4. Stamp the ingest state:
   ```bash
   npx tsx scripts/stamp-ingest.ts update-culture --summary "НФЦ film register: <N> films, €<X>M, <P> producers, <first>–<last>"
   ```

## Notes

- Recipients are keyed by **producer name** (the register has no EIK) — grouped by a
  normalised `producerFold`. One name may span related companies; never assert a
  person↔company link on a name alone (plan §6).
- The one-off nature of the .xls set means a full re-download is cheap; there is no
  incremental backfill flag beyond `--force`.