git:20261014.23eab7e to git:20261014.c73e656

---

name: cognee-migrations

description: Use when dealing with cognee database migrations — understanding when they run automatically, checking or repairing migration state with cognee-cli upgrade/downgrade/stamp/current, a write blocked by a failed migration, authoring a new Alembic (relational schema) revision or a graph/vector data migration, or moving data between systems (relational DB import, memory export/import).

---

# Database migrations

cognee has **two migration chains**, run together:

| Chain | Changes | Lives in | Revision stored in |

|---|---|---|---|

| **Relational schema** (Alembic) | Tables and columns of the relational DB (users, datasets, ACLs, pipeline runs, …) | `cognee/alembic/` (`alembic.ini` is in `cognee/`) | `alembic_version` table |

| **Graph/vector data** | Cross-store data rewrites (re-keying node ids, adding graph columns) across graph DB, vector DB and relational ledger | `cognee/modules/migrations/` (`registry.py`, `versions/`) | Per dataset: `dataset_database.migration_revision` (access control on). Globally: `global_database_version.global_migration_revision` (access control off) |

## Use it

### They run by themselves

`run_migrations()` applies the relational chain first, then the data chain.

It runs:

- at API server startup (and in the Docker `entrypoint.sh` before gunicorn);

- on the first write in an SDK or CLI process (`remember`, `add`, `cognify`,

`improve`, `memify`, memory imports), once per process;

- when you call `await cognee.run_migrations()`.

A fresh database is built by running the whole chain (no stamping). At

head, the first run in each process still does a no-op Alembic upgrade plus

a scan of the per-database revision rows; later calls in the same process

are skipped by an in-memory flag. `ENABLE_AUTO_MIGRATIONS=false`

turns off all automatic runs; then run `cognee-cli upgrade` yourself.

Concurrent processes are serialized by a migration lock: a Postgres advisory

lock (works across hosts) or a file lock next to the SQLite DB (one host

only).

### Check and repair

```bash

cognee-cli current # stamped revision per database, and the last failure

cognee-cli history # the data chain, newest first

cognee-cli upgrade # relational to head, then data chain to head

cognee-cli upgrade <slug> # data chain up to and including <slug>

cognee-cli upgrade --alembic <rev> # pin the relational target

cognee-cli downgrade <slug|base> [--dataset UUID ...] [--alembic REV] [--force]

cognee-cli stamp <head|base|slug> [--dataset UUID ...] [--force]

```

- The positional revision is always a **data-chain slug**; relational

targets go through `--alembic`.

- `downgrade` rewrites data and asks for confirmation. It only reverts

spans where every migration defines a `down()`, and leaves the relational

schema alone unless you pass `--alembic`.

- `stamp` changes only the stored data-chain revision, without running

anything. Use `stamp base --dataset <id>` and then `upgrade` when a

database's data drifted from its stamp (for example after restoring a

backup); the chain is idempotent and converges it.

- `upgrade` runs even with `ENABLE_AUTO_MIGRATIONS=false`.

### A write is blocked

If a dataset's data migration failed, writes to that dataset are refused

until it succeeds (with access control off, any failure blocks all writes).

The server still starts. Run `cognee-cli current` to see the error, fix the

cause, then `cognee-cli upgrade`. A failed run is retried on the next start

or write.

### Moving data between systems (not schema migrations)

| Goal | Use |

|---|---|

| Turn an existing relational database into a graph | `migrate_relational_database(graph_db, schema)` (`cognee/tasks/ingestion/migrate_relational_database.py`), with the source DB set by `MIGRATION_DB_PROVIDER` / `_PATH` / `_NAME` / `_HOST` / `_PORT` / `_USERNAME` / `_PASSWORD`. Examples: `examples/demos/ingestion_and_migration/` |

| Back up a dataset, or move it to another cognee instance | A COGX archive, see below |

| Export a dataset's graph for other tools | `await cognee.export(dataset, format=...)`: `"json"`, `"graphml"` or `"cypher"` write a file (one way: cognee can't import them back); `"pydantic"` (default) returns typed DataPoint objects in memory |

| Import from another memory system (Mem0, Zep/Graphiti, Letta, LangMem) | Build a `MemorySource` (`cognee/modules/migration/sources/`) and pass it to `await cognee.remember(source, dataset_name=...)` |

There is no tool that moves a whole deployment from one database backend to

another.

#### COGX archives

COGX (Cognee eXchange, `cognee/modules/migration/cogx.py`) is cognee's

portable memory format and the only export format cognee can import back.

An archive is a directory with a `manifest.json` (COGX version, source

system, the dataset's data-migration revision) and one JSONL file per record

kind (`documents`, `episodes`, `entities`, `facts`, `memories`,

`memory_blocks`), plus `nodes.jsonl` with the raw graph nodes. The Mem0,

Zep, Letta and LangMem importers also translate into COGX records first.

Since COGX 0.2 every record also carries an optional OMP header, the fields

proposed to the Open Memory Protocol exchange group: `origin` (first system

and id, kept across copies), `content_hash` (sha256 of the body, for dedupe;

distinct from an integrity checksum), `subject`, `semantic_kind` (fact,

decision, commitment, procedure, preference, observation, other),

`valid_from`/`valid_to` with `invalidated_by`, `evidence[]` (role, ref,

`version`, `content_type` (the body's format), `tags`, a record `checksum`

stamped by the writer, and provenance as `contributors[]` (role, actor,

kind, at) with the `accountable` human and `ai_involvement`. The

cognee exporter fills what it can read from the graph and declares the rest

in the manifest's `coverage` map; the loader keeps the header in each

record's external metadata and on fact edges, so a re-export preserves it.

0.1 archives read unchanged.

Use it to back up and restore a dataset, or to copy one to another cognee

instance:

```python

from cognee.migration import COGXArchiveSource

await cognee.export("my_dataset", format="cogx", destination="backup_cogx")

await cognee.remember(COGXArchiveSource("backup_cogx"), dataset_name="my_dataset")

```

- A restore defaults to `mode="preserve"`: the archived graph is written

back as-is, with no LLM calls. `mode="hybrid"` also re-cognifies the raw

content; `mode="re-derive"` ignores the archived graph and extracts again

(costs LLM tokens).

- `cognee.push()` / `cognee-cli push` does the same to Cognee Cloud: it

exports to COGX, packs it as a `.cogx.tar.gz` and uploads it, and the

receiving instance restores it (preserve mode unless you pass `mode=`).

- `export(..., include_permissions=True)` also writes `permissions.json`

with the dataset owner and ACL grants, **including password hashes**, so

the restore recreates working accounts. Treat that archive as a secret.

- An archive written by a newer major COGX version is rejected with a

`ValueError`; upgrade cognee on the importing side.

## Pitfalls

- **Never regenerate `cognee/alembic/frozen_schema.py`.** It is the

certified base schema the initial revision builds from, pinned by

`cognee/tests/unit/test_frozen_schema_seal.py`. Schema changes ship as

new revisions at head.

- **Never hand-type an Alembic revision id.** Hand-typed patterns

(`a1b2c3d4e5f6`, …) already collided with a downstream chain that vendors

this one (`b2c3d4e5f6a7`). Generate ids with `alembic revision`.

- **Never rename, remove, or reorder a data-chain entry.** The slug is what

deployed databases store; an unknown stored slug disables the chain for

that database.

- **A model change and its migration land together.** CI's "Migration/Model

Lockstep Guard" fails otherwise.

- **The relational schema cannot be downgraded below the data-bookkeeping

revisions** unless the data chain goes to `base` in the same call.

- `alembic.ini` is always the packaged one; `COGNEE_ALEMBIC_PATH` or

`--alembic-path` only changes the scripts directory (for vendored chains).

## How it works

`run_migrations()` (`cognee/modules/migrations/startup.py`) takes the

migration lock, decides fresh vs existing (a `users` or `alembic_version`

table exists), runs Alembic in-process on a worker thread, then walks the

data chain per database with `runner.run_database_migrations`, stamping

after every step. After the chain, it syncs vector-adapter storage when the

recorded `cognee_version` differs from the library's

(`versions/adapter_storage_migration.py`, not a chain entry).

- Relational: `cognee/alembic.ini`, `cognee/alembic/env.py`,

`cognee/alembic/versions/`, `cognee/alembic/frozen_schema.py`

- Data chain: `cognee/modules/migrations/` (`README.md` is the authoring

contract; `registry.py`, `migration.py`, `runner.py`, `startup.py`,

`versions/`)

- CLI: `cognee/cli/commands/migrate_command.py`

## Extending it

### A new Alembic revision

```bash

cd cognee # the directory with alembic.ini

uv run alembic revision -m "add foo to data"

```

The DB URL comes from the live relational engine, so your normal `.env`

settings apply. Follow the recent revisions (for example

`versions/e7f9a1c3d5b8_add_data_dataset_created_index.py`):

- **Idempotent and guarded:** inspect first (`sa.inspect(op.get_bind())`)

and skip when the table is missing or the column/index already exists.

- **Branch on dialect** (`conn.dialect.name == "postgresql"`) for

Postgres-only SQL. For Postgres enums use `postgresql.ENUM(...,

create_type=False)` and create the type up front with `checkfirst`.

- **SQLite cannot drop or alter columns in place:** use

`op.batch_alter_table(...)`.

- **Indexes:** plain `CREATE INDEX IF NOT EXISTS` inside the migration

transaction, not `CONCURRENTLY` (which releases the version-row lock and

lets concurrent workers into the same build). Repair invalid Postgres

indexes via `pg_index.indisvalid`.

- A new model module outside the usual import path must be imported in

`cognee/alembic/env.py`, or autogenerate will not see it.

### A new data migration

Read `cognee/modules/migrations/README.md` first. In short:

1. Write a module in `cognee/modules/migrations/versions/` with

`async def migrate(context)` (and optionally `async def downgrade(context)`);

step 2 registers them as `up=` / `down=`.

2. Append `Migration(slug=..., cognee_version=..., up=..., down_revision=<previous slug>, down=...)`

to `MIGRATIONS` in `registry.py`. The chain is validated at import

(linear, unique slugs).

3. Make it idempotent and cheap on empty stores, and crash-safe: re-key

derived stores (vectors, ledger) first and rename in the graph last.

4. Freeze private copies of any logic you depend on; never import live

models that may change later.

### Tests

- `cognee/tests/e2e/migrations/test_migration_model_lockstep.py` (+

`schema_baseline.json`): the lockstep CI job.

- `cognee/tests/unit/test_run_migrations.py`: single Alembic head, startup

behaviour.

- `cognee/tests/unit/test_frozen_schema_seal.py`: frozen schema fingerprint.

- `cognee/tests/unit/modules/migrations/`: data-chain unit tests.

- `cognee/tests/migrations/test_migration_lockstep.py`: data chain against

real stores (seed, down, up, verify).