mongodb · git:20260602.a962997 · 2026-06-02 · sha256 fe167a8986886fc4
mongodb git:20260602.a962997A
Immutable. This exact content never changes and is served at /api/v1/blob/fe167a8986886fc4.
---
name: mongodb
description: "Use when modeling MongoDB documents (embed vs reference, the 16MB cap, bucket/subset/extended-reference patterns), choosing or fixing indexes (compound order via the ESR rule, partial/TTL/multikey, reading explain), writing aggregation pipelines that stay index-eligible ($match-first, $lookup cost, $group memory + allowDiskUse), running multi-document transactions with retry, or operating/securing a deployment (replica set, read/write concern, Atlas M0/Flex/dedicated tiers, Vector Search, Queryable Encryption). Triggers: 'slow Mongo query', 'add an index', 'COLLSCAN', 'embed or reference', 'aggregation pipeline', '$lookup', 'transaction keeps retrying', 'modela esta colección', 'mi consulta hace un full scan'. MongoDB 8.2, driver-agnostic (mongosh syntax). NOT relational schema/SQL/EXPLAIN ANALYZE (that is postgresdb)."
tags: [mongodb, nosql, database, aggregation, indexing]
recommends: [secure-coding, harness]
profiles: []
origin: risco
---
# MongoDB — modeling, indexing, aggregation, transactions, ops
Engine-level MongoDB 8.2 guidance: model documents for the queries you actually run, pick the
index the planner will use, write aggregation pipelines that stay index-eligible, run
multi-document transactions with correct retry, and operate/secure a deployment. Driver-agnostic —
every example is `mongosh` shell syntax that maps 1:1 to the official drivers (Node, Python, Go,
Java, Rust). This skill owns the **server's query and the index it picks**, not any ODM's API.
## When to use / When NOT to use
**When to use:**
- Document modeling: embed vs reference, the 16 MB cap, one-to-many/many-to-many, the
subset/extended-reference/bucket/computed/outlier patterns, taming unbounded array growth.
- Index decisions: single-field, compound (the ESR ordering rule), multikey, partial, TTL, text,
wildcard, `2dsphere`; and when an index is NOT worth it.
- Any query that is slow or scans too much; reading `explain("executionStats")`.
- Aggregation pipelines: stage order so `$match`/`$sort` hit an index, `$lookup` cost, `$unwind`
explosion, `$group`/`$sort` memory limits and `allowDiskUse`, `$merge`/`$out`, faceting.
- Multi-document transactions: sessions, `withTransaction` retry semantics, read/write concern.
- Operating/securing: replica set, read preference, write concern, Atlas tier choice, Atlas Search
& Vector Search, Queryable Encryption, role-based access, connection-pool knobs.
**When NOT to use:**
- Relational schema / SQL / `EXPLAIN ANALYZE` → [`postgresdb`](../postgresdb/SKILL.md). Different
engine, planner, and concurrency model.
- ODM/driver API ergonomics (Mongoose pre-save hooks, the Node driver's `bulkWrite` return shape,
`updateMany`'s result object) → that tool's own docs. This skill owns the server query and the
index the server picks, not the JS object the driver hands back.
- App-layer caching as a product (Redis-in-front-of-reads).
- Cloud-console click-paths — we give the shell command / connection string, not the Atlas UI tour.
- Picking a vector store *across engines* (Pinecone vs Weaviate). Atlas Vector Search *inside* Mongo
is in scope; cross-engine selection is not.
Deep dives: [data-modeling](references/data-modeling.md) (embed/reference tree, all six patterns,
16 MB math, polymorphic & schema versioning) · [aggregation](references/aggregation.md) (per-stage
index eligibility, `$lookup` variants, `$facet`, window fns, `$merge`/`$out`, reading pipeline
`explain`) · [transactions-and-ops](references/transactions-and-ops.md) (retry wrappers, concern
semantics, Atlas tiers, Vector Search, Queryable Encryption, RBAC, pooling, change streams).
## Non-negotiables
1. **Design for the queries you run, not the shape of your data.** The schema is the set of
documents that make your common reads single-document and index-eligible.
2. **Never let an array grow unbounded inside a document.** It walks toward the 16 MB cap, bloats
every read of the parent, and kills update performance — reference or bucket it.
3. **The hard ceiling is 16 MB per document.** If a one-to-many can exceed it, you reference; there
is no `TOAST`-style overflow here.
4. **A single-document write is already atomic.** Reach for a multi-document transaction *only* when
two or more documents must change together — otherwise you are paying for nothing.
5. **Every transaction retries on the `TransientTransactionError` label** (and commit retries on
`UnknownTransactionCommitResult`). `withTransaction` does both for you; a hand-rolled loop must.
6. **Index by ESR: Equality fields, then the Sort field, then Range fields.** This order lets one
compound index serve the filter, the sort, and the range without an in-memory sort.
7. **Read `explain("executionStats")` before and after adding an index** — confirm `IXSCAN`, not
`COLLSCAN`, and `totalKeysExamined ≈ nReturned`. Or it didn't happen.
8. **`w:"majority"` for money and state transitions**, read concern `"majority"`/`"snapshot"` when a
read must reflect a durable write. `w:1` can be rolled back on a primary failover.
9. **Money is `Decimal128` (`NumberDecimal("...")`), never a JS `double`.** Binary floats drift;
`0.1 + 0.2 !== 0.3` in your ledger.
10. **Never store a secret in plaintext.** Use Queryable Encryption / client-side field-level
encryption; never commit a `mongodb://user:pass@` literal.
## Decision rules
### Embed or reference
| Relationship | Choose | Why |
| --- | --- | --- |
| Read together, small, bounded (address on a user) | embed | one read, no `$lookup`, atomic update |
| One-to-few, bounded (≤ a few dozen, won't grow) | embed | stays well under 16 MB |
| One-to-many, growth not bounded (comments on a post) | reference | array would chase the 16 MB cap |
| Many-to-many (students↔courses) | reference (array of ids on the lighter side) | shared, independently mutated |
| Child shared across parents | reference | one source of truth, no duplication drift |
| Child independently and frequently mutated | reference | avoid rewriting a big parent per child edit |
| High-cardinality / huge child set | reference (+ optional subset embed) | keep the hot read small |
### Which schema pattern
| Symptom | Pattern | What it does |
| --- | --- | --- |
| List view reads 3 fields of a heavy doc | **subset** | embed only the hot fields, reference the rest |
| `$lookup` on every read just to show a name/price | **extended reference** | copy the few joined fields you display |
| Unbounded time-ordered events (readings, logs) | **bucket** | group N events per doc by time window |
| Same `count`/`sum` recomputed on every read | **computed** | store the rollup, update it on write |
| 1% of docs break the shape (a few mega-children) | **outlier** | flag them, overflow into linked docs |
| One collection holds several entity shapes | **polymorphic** | a `type` discriminator + shared `_id` space |
Full Bad→Good documents for each in [data-modeling](references/data-modeling.md).
### Which index type
| Access pattern | Index | Note |
| --- | --- | --- |
| `=` on one field | single-field | also covers the field's sort |
| filter + sort + range together | compound, ordered **ESR** | one index serves all three |
| query into an array field | multikey (automatic on an array key) | one multikey field per compound index |
| query only a subset of docs (`status:"active"`) | partial (`partialFilterExpression`) | smaller, cheaper to maintain |
| auto-expire docs after a time | TTL (`expireAfterSeconds` on a Date) | single-field only; deletes in background |
| language-aware text search | text **or** Atlas Search | Atlas Search is far richer; text is legacy |
| unpredictable / many query shapes on subdocs | wildcard (`"$**"`) | last resort; never beats a targeted index |
| geospatial proximity / within | `2dsphere` | GeoJSON `Point`/`Polygon` |
| vector similarity (8.2, Community+) | Atlas/Vector Search index | see transactions-and-ops ref |
### When NOT to add an index
- Low-cardinality field (a boolean, a 3-value status) — the planner skips it; `COLLSCAN` wins.
- Tiny collection — a collection scan reads one or two pages; the index is pure write tax.
- A field already the **left prefix** of an existing compound index — redundant.
- Write-heavy field rarely filtered — every index is paid on every insert/update.
- "Just in case" indexes — an unused index costs writes and RAM, returns nothing.
## Copy-paste patterns
Every fence is `mongosh` syntax.
### Model the document for the read (Bad → Good)
```javascript
// BAD: comments embedded in the post — array grows without bound toward 16 MB,
// every post read drags the entire comment history, money is a float.
db.posts.insertOne({
_id: ObjectId(),
title: "Indexing 101",
authorId: ObjectId(),
price: 9.99, // double — drifts in arithmetic
comments: [ /* ...unbounded... */ ] // chases the 16 MB cap
})
// GOOD: post stays small; comments referenced; money is Decimal128;
// the few fields the feed needs are duplicated (extended reference).
db.posts.insertOne({
_id: ObjectId(),
title: "Indexing 101",
author: { _id: ObjectId(), name: "Ada" }, // extended ref: name shown without a $lookup
price: NumberDecimal("9.99"),
commentCount: 0, // computed rollup, bumped on write
createdAt: new Date()
})
db.comments.insertOne({ _id: ObjectId(), postId: ObjectId(), body: "…", createdAt: new Date() })
```
### Compound index in ESR order + the query that uses it
```javascript
// Feed query: filter by author (equality), sort by date (sort), bound by a date (range).
// ESR => author first, then the sort/range key.
db.posts.createIndex({ "author._id": 1, createdAt: -1 })
db.posts.find({ "author._id": authorId, createdAt: { $gte: since } })
.sort({ createdAt: -1 })
.limit(20)
// Confirm the plan: IXSCAN on the index above, no in-memory SORT stage.
```
### Partial + TTL indexes
```javascript
// Partial: index only the rows you actually query (active orders), not the archive.
db.orders.createIndex(
{ customerId: 1, createdAt: -1 },
{ partialFilterExpression: { status: "active" } }
)
// TTL: expire sessions 30 minutes after lastSeen. Field MUST be a Date.
db.sessions.createIndex({ lastSeen: 1 }, { expireAfterSeconds: 1800 })
```
### Aggregation: `$match` first, `$lookup`, `$group` with `allowDiskUse`
```javascript
db.orders.aggregate([
// $match FIRST so it uses the compound index and shrinks the working set early.
{ $match: { status: "paid", createdAt: { $gte: since } } },
{ $sort: { createdAt: -1 } }, // index-eligible here, before any $group/$project
{ $lookup: {
from: "customers",
localField: "customerId",
foreignField: "_id",
as: "customer",
pipeline: [ { $project: { name: 1 } } ] // project inside $lookup: pull only what you need
}},
{ $group: { _id: "$customerId", total: { $sum: "$amount" } } }
], { allowDiskUse: true }) // $group/$sort spill past 100 MB/stage; this lets large groups complete,
// it is NOT a substitute for a missing $match index — see anti-patterns.
```
### Read `explain("executionStats")` — the four numbers
```javascript
db.posts.find({ "author._id": authorId }).sort({ createdAt: -1 })
.explain("executionStats")
```
Read these before declaring a fix:
1. **`winningPlan.stage`** — must be `IXSCAN` (or `FETCH`→`IXSCAN`), not `COLLSCAN`.
2. **`totalKeysExamined` vs `nReturned`** — close means the index is selective; a huge ratio means
the index scans far more than it returns (wrong key order, low selectivity).
3. **A `SORT` stage** — an in-memory sort the index should have satisfied; reorder by ESR to remove it.
4. **`rejectedPlans`** — what the planner considered and dropped; a near-miss hints at a better index.
### Multi-document transaction with full retry
```javascript
// Use withTransaction — it retries the body on TransientTransactionError and retries the
// commit on UnknownTransactionCommitResult for you. Requires a replica set / sharded cluster.
const session = db.getMongo().startSession();
try {
session.withTransaction(() => {
const orders = session.getDatabase("shop").orders;
const ledger = session.getDatabase("shop").ledger;
orders.updateOne({ _id: orderId, status: "pending" }, { $set: { status: "paid" } }, { session });
ledger.insertOne({ orderId, amount: NumberDecimal("9.99"), at: new Date() }, { session });
}, { readConcern: { level: "snapshot" }, writeConcern: { w: "majority" } });
} finally {
session.endSession();
}
// If both writes target ONE document, drop the transaction — that write is already atomic.
```
### `bulkWrite` upsert
```javascript
db.inventory.bulkWrite([
{ updateOne: {
filter: { sku: "ABC-1" },
update: { $inc: { qty: 5 }, $setOnInsert: { createdAt: new Date() } },
upsert: true
}}
], { ordered: false }) // ordered:false keeps going past one failed op and parallelizes
```
### Change stream (resumable tail)
```javascript
// Watch only the events you care about; persist resumeToken to restart without gaps.
const cs = db.orders.watch([{ $match: { operationType: { $in: ["insert", "update"] } } }]);
while (cs.hasNext()) { const change = cs.next(); /* process; save change._id as resume token */ }
```
More variants (`$facet`, window functions, `$merge`/`$out`, vector search) live in the references.
## Anti-patterns / rationalizations → STOP
| Rationalization | Reality → STOP |
| --- | --- |
| "Embed all the comments, it's one read" | Array grows unbounded toward 16 MB and bloats every post read. Reference or bucket. |
| "`$lookup` is just a JOIN, use it everywhere" | Mongo is not relational; per-document `$lookup` is expensive. Prefer modeling (extended reference) so the read needs no join. |
| "Wrap this single-document update in a transaction to be safe" | A single-doc write is already atomic. The transaction adds latency and a replica-set requirement for zero gain. |
| "`COLLSCAN` is fine, it's fast on my 100 docs" | It is O(n); at 4M docs it is a full table read. Add the index now and prove `IXSCAN`. |
| "Set `allowDiskUse:true` and the slow pipeline is fixed" | That masks a missing `$match` index by spilling to disk. Fix stage order / add the index first. |
| "Store the price as a number, round on display" | JS doubles drift across `$sum`/`$inc`. Use `NumberDecimal` (`Decimal128`). |
| "Group the whole collection, no `$match`" | A blocking `$group` over everything blows the 100 MB/stage limit. `$match` first to shrink it. |
| "One collection for users, orders, logs — fewer to manage" | Mixed shapes kill index selectivity and balloon working set. Split by access pattern. |
| "`$where` lets me run a quick JS predicate" | Runs JS per document, no index, a server-side injection surface. Use query operators / `$expr`. |
| "Index every field just in case" | Each index is a write tax and RAM cost; unused indexes return nothing. Index for real query shapes only. |
## Quick reference
### Read/write concern matrix
| Need | Write concern | Read concern | Note |
| --- | --- | --- | --- |
| Money / state transition | `w:"majority"` | `"majority"` | survives a primary failover |
| Read your own durable write | `w:"majority"` | `"majority"` (+ causal session) | no rollback window |
| Transaction default | `w:"majority"` | `"snapshot"` | consistent point-in-time |
| Logs / fire-and-forget | `w:1` | `"local"` | fast, may be rolled back |
### Atlas tier chooser
| Tier | Use it for | Limits |
| --- | --- | --- |
| **M0** | learning, tiny prototypes | free forever, up to 5 GB, shared, no SLA |
| **Flex** (GA Feb 2025) | small prod / variable load | $8 base capped at $30/mo, 100 ops/sec (burst 500), 5 GB; supports Atlas Search, Vector Search, Change Streams, Triggers |
| **M10+** (dedicated) | production, isolation, scale-up | from ~$0.08/hr (~$57/mo); dedicated resources, full features |
M0 does **not** run Vector Search well for real workloads — move to Flex or dedicated. Legacy
Serverless / M2 / M5 were auto-migrated to Flex.
### Aggregation memory
Each blocking stage (`$group`, `$sort` without an index, `$bucket`) is capped at **100 MB**. Past it
the stage errors unless `allowDiskUse:true` lets it spill. Spilling is a correctness fallback for
genuinely large groups, not a performance fix for a missing index.
## Verify
Run `scripts/verify.sh` from your project root. It is read-only, never connects to a database, and
never writes. It scans discovered `.js`/`.mongodb.js` files and flags foot-guns: a committed
plaintext `mongodb://user:pass@` credential (the **only** hard failure), `createIndex` calls with no
options, redundant compound-index prefixes, `$where` predicates, unbounded `$lookup`,
`allowDiskUse:true` that may be masking a missing index, and money stored as a JS number in seed
scripts. If `node` is present it runs `node --check` for a syntax pass; otherwise that step is
`[skip]`. Everything except a committed credential is advisory `[warn]`/`[skip]`. It runs on stock
macOS bash 3.2 and exits 0 on a clean or empty target.
## Project grounding (02-DOCS + CLAUDE.md)
When this skill runs in a project with a `02-DOCS/` layer (the [`harness`](../harness/SKILL.md)
Karpathy wiki), record this project's MongoDB decisions there and index them from the root
`CLAUDE.md`, so the next agent inherits the conventions instead of re-deriving them.
1. **Find the article** `02-DOCS/wiki/stack/mongodb.md`, linked from a `## Knowledge map` section in
the root `CLAUDE.md`.
2. **If missing or stale**, create/update it with the project's real choices — collection layout and
embed/reference decisions, the index set and its ESR rationale, read/write concern policy, the
Atlas tier, and any encryption/RBAC setup — then add/refresh the `CLAUDE.md` link (create the
`## Knowledge map` section, and `CLAUDE.md` itself, if absent).
3. **Read it first on every use** and stay consistent; when a convention changes, update the article
(bump its `Updated` date) in the same change.
No `02-DOCS/` layer? Skip silently (optionally suggest `harness`). Technical conventions are
*recorded, not gated* — never block the task on this.
## See Also
- [references/data-modeling.md](references/data-modeling.md) — embed/reference tree, the six
patterns with worked documents, 16 MB math, polymorphic & schema versioning.
- [references/aggregation.md](references/aggregation.md) — per-stage index eligibility, `$lookup`
variants, `$facet`, window functions, `$merge`/`$out`, hybrid `$scoreFusion`, reading pipeline
`explain`.
- [references/transactions-and-ops.md](references/transactions-and-ops.md) — retry wrappers, concern
semantics, replica-set requirement, Atlas tiers, Search/Vector Search, Queryable Encryption, RBAC,
pooling, change streams.
- Sibling skills: [`harness`](../harness/SKILL.md) (scaffolds the `01-TOOLS/MONGODB` operational
tool) and [`secure-coding`](../secure-coding/SKILL.md) (auth, encryption, least-privilege).
- For relational work — SQL, foreign keys, `EXPLAIN ANALYZE`, MVCC — use
[`postgresdb`](../postgresdb/SKILL.md), not this skill. Different engine and planner.
- Out of scope here — external tools with their own docs: ODM/driver API surface (Mongoose hooks,
the Node driver's `bulkWrite`/`updateMany` return shapes) and cross-engine vector-store selection.
This skill owns the server query and the index the server picks.