git:20260906.e6ad4bd to git:20260911.37b054b
31 added, 17 removed. Audit A to A.
---
name: cache-sharding-and-replication
description: >
Topology for a cache that no longer fits one node: client-side sharded, proxy-fronted,
clustered, and fully replicated, compared on failure behaviour, cost and client
complexity; and why a read after a write on a replicated cache is not read-your-writes.
Estimates origin load when a cache node fails from its measured request share and the
surviving copies, routing and capacity — mitigated by
replication, warming, coalescing and admission control. Use when choosing between client
sharding, a proxy and cluster mode, when a cache node loss or rolling restart took the
database with it, when replicas of a cache disagree, or when deciding between sharding the
cache and replicating all of it. Does not cover whether to cache, TTL, stampede or
invalidation (caching-strategies), the key-to-node mapping (consistent-hashing), a single
hot cache key (hot-partitions-and-rebalancing), entry serialisation cost
(serialization-performance), or what a replicated read observes (consistency-models).
---
# Cache Sharding And Replication
## Purpose
Decide how a cache is laid out across nodes, and what happens when one of those nodes goes
away. This is a topology skill only: whether to cache, how long to keep an entry, and how to
invalidate it are `caching-strategies`, and everything here assumes those decisions are
already made.
The failure this prevents is the one that never looks like a cache incident. A cache node is
restarted for a routine upgrade; with N balanced nodes, consistent hashing and no replicas,
about 1/N of the keyspace loses its cached copy. After fail-fast remapping, requests for those
keys can reach the origin until refill, while survivors continue serving. The cache tier
reports a modest dip in hit rate. The database saturates. Nobody investigating the database
is looking at the cache, because the cache is up.
## Workflow
- Before recommending changes, inspect cache product/version, Java client and resolved dependencies,
- runtime/toolchain, routing and retry configuration, replica placement, working-set bytes, per-node
- request share and origin capacity at the required SLO. This skill is language-independent and
+ Before recommending changes, reuse the request, configuration and existing measurements: cache
+ product/version, relevant clients and runtimes, routing/retry configuration, replica placement,
+ working-set bytes, per-node request share and origin capacity at the required SLO. Ask only about
+ unresolved constraints that change the decision, such as permitted stale reads or replica locations;
+ continue independent analysis with explicit assumptions. This skill is language-independent and
declares no Java baseline or executable Java examples; do not infer support for a client feature
or authorize an upgrade. With missing measurements, provide conditional estimates and the exact
measurement needed, not a production sizing or confirmed incident diagnosis.
1. **Classify the cache first: performance or availability.** If the origin cannot serve the
full request rate with the cache empty, the cache is an availability component. Separate this
from durability: the cache does not become the authoritative data store by being essential.
2. **Do the node-loss arithmetic before choosing a topology.** Measure the request share owned by
each node; `total_rate / N` is only the uniform approximation. Losing node `i` can send its
request share to the origin, plus secondary evictions and retries. Compare rate, concurrency,
query mix and duration to the origin's measured capacity. The worked example is
`references/node-loss-and-origin-protection.md`.
3. **Choose sharding or full replication from the working set.** If the whole working set
fits comfortably in one node's memory and reads dominate, replicating everything can remove a
network hop only when the replica is process-local. It avoids key loss after a node failure if
routing and remaining capacity work, and costs roughly `N ×` value memory plus metadata.
4. **Choose the topology** — client-sharded, proxy, or clustered — on operational cost and
client complexity and measured end-to-end latency. The comparison is `references/topologies.md`.
5. **Set the replication factor from step 2**, not from a default. Replication exists here to
keep the shard served when a node dies; if the arithmetic says the origin survives a node
loss, RF=1 is a legitimate, cheaper answer. RF counts all copies including the primary;
place them across the failure domains being protected and check promotion/quorum requirements.
6. **Exercise failure under load**—crash, partition/timeout, promotion and rejoin—and assert bounds
on origin rate/concurrency, client errors and recovery, not only cache hit rate.
7. **Add a local L1 only for a measured reason**, and accept that invalidation now has to
reach every instance's L1 as well as the shared tier.
## Decision block
```text
Sharded cache (each key has an owner shard, with optional replicas) when:
- the working set exceeds one node's memory, or memory cost makes N copies unattractive
- writes and invalidations are frequent enough that keeping N copies converged is work
Fully replicated cache (every node holds everything) when:
- the working set fits one node's memory with headroom, reads dominate heavily, and the
value of surviving node loss exceeds N × memory; only process-local copies remove the hop
- typically the shape of small reference data: feature flags, rates, configuration
Replicate each shard (RF > 1) when:
- the measured node-loss arithmetic exceeds origin headroom
- or one shard is read-hot and the product can route reads to replicas within the required
consistency model
Keep RF = 1 when:
- the origin demonstrably absorbs a node loss, and the memory is better spent on a larger
working set; replicas can also serve reads if the product and consistency contract allow it
Prefer a proxy or a clustered cache over client-side sharding when:
- - clients are polyglot, numerous, or cannot be redeployed together; the topology then
- changes without touching them
+ - its routing/discovery protocol reduces coordination across numerous or polyglot clients;
+ check actual client capabilities, because dynamic client-side membership can also avoid deploys
Prefer client-side sharding when:
- clients are few and share a runtime, and the extra network hop is a measurable share of
the cache's own latency — the point of a cache is that it is fast
Do not add a cache node to fix a hot key:
- one key has one owner under every mapping function (hot-partitions-and-rebalancing)
```
## Rules
- **Losing a cache node is an origin-load event.** Size the origin, or the protection in
front of it, for the loss of one cache node — that is a routine occurrence (upgrade,
eviction, spot reclaim), not a disaster scenario.
- A rolling restart can cause repeated remapping or replica promotion. Gate each next restart
on origin headroom, client SLOs and restored replica readiness, not just recovered hit rate.
- - Stable placement limits movement: `hash(key) % N` remaps
- nearly the entire keyspace on a membership change, turning one node's loss into a total
- miss storm. Consistent hashing is one option; fixed slots can also preserve placement.
- The mapping function belongs to `consistent-hashing`; this is the consequence.
+ - Stable placement limits movement: changing N in `hash(key) % N` can remap a large fraction,
+ depending on the divisors and node-index mapping. The moved keys' request share, surviving
+ copies and refill policy determine miss traffic; compare resulting origin demand with measured
+ capacity before predicting an outage. Consistent hashing is one option; fixed slots can also
+ preserve placement. The mapping function belongs to `consistent-hashing`.
- Node loss has a **second-order** cost when keys remap and refill on survivors: their
memory did not grow, so insufficient headroom can raise evictions on previously healthy shards.
Measure this effect rather than assuming the hit-rate dip equals the lost key share.
- **Replication alone does not give read-your-writes.** With asynchronous replication, a write
acknowledged by one replica and a read served by another may return the old value. If the requirement is that a
user sees their own change, route to a copy known to have applied that write, or bypass stale
cache copies and read an origin endpoint that supplies the guarantee. Arbitrary replica affinity
and invalidation alone are insufficient; promotion may lose an acknowledged write.
`consistency-models` owns the guarantee, including the behavior across failover.
- Replication guarantees are product/configuration-specific. Asynchronous replicas have no useful
convergence deadline unless lag is bounded and monitored. TTL bounds how long a missed
invalidation can survive only if expiry forces a correct reload; it is not a consistency proof.
- Client-side sharding puts the topology in every client. Adding a node means every client
must use compatible node lists, virtual-node counts and hashes; overlapping versions need
an explicit migration/invalidation protocol. A
disagreement is two clients writing the same key to two different nodes, and both of them
may read stale. Distribute membership through one source, versioned; versioning alone does
not make an overlapping rollout coherent.
+ - Replica and L1 placement must respect data-access and residency constraints. Preserve tenant
+ and authorization distinctions in keys and access checks; a key prefix is not access control.
+ Check whether one tenant's refill can consume the shared origin budget before widening replication.
- A proxy costs one extra network hop on the cache path, which is the path chosen for being
fast. Measure the hop against `T_source` before rejecting it: a fraction of a millisecond
in front of a source costing tens of milliseconds is usually the right trade, and it buys
topology changes without client deploys.
- A clustered cache with server-owned placement moves membership out of application config.
Cross-slot multi-key semantics vary by product; Redis Cluster rejects many such operations,
while other systems coordinate them at extra latency/availability cost. Check the exact command
and failure contract against the access pattern.
- **A near-cache (local L1 in front of the shared L2) is a second cache with its own
- coherence problem**, and it is per-instance: invalidating the L2 invalidates no L1.
- `caching-strategies` owns invalidation propagation and the L1 TTL as the safety net; the
- topology consequence is that the copies to invalidate now number `instances + replicas`.
+ coherence problem**, and it is per-instance. An L2 acknowledgement alone does not establish
+ that every L1 is current; a product may propagate invalidations, with delivery/loss-recovery
+ semantics to verify. `caching-strategies` owns that protocol and the source-age budget across
+ layers; the topology consequence is up to `instances caching the key + RF` copies per key,
+ excluding temporary migration copies.
- Every entry crossing the network is serialised, so the value size is a throughput decision,
not a detail. A large value multiplied by the fan-out of a warm-up is a network incident —
`serialization-performance` owns the format cost.
## Deliverable
- Return the chosen topology and rejected alternative, measured inputs versus assumptions,
- node-loss origin-load estimate, replica placement/read policy, and failure-test acceptance
- bounds with rollout abort criteria. For incidents, distinguish observed timing/counters from
- the cache-loss hypothesis and name the load test or evidence that would refute it. State which
- checks actually ran; a paper estimate is not demonstrated failure tolerance.
+ Scale the result to the task. For a topology decision, return the recommendation (including
+ keeping the current design), the materially relevant alternative and why it loses, measured
+ inputs versus assumptions, node-loss origin-load estimate, replica placement/read policy, and
+ failure-test acceptance bounds with rollout abort criteria. State what evidence would change
+ the decision. Stop discovery when the remaining unknowns do not change it; otherwise name the
+ specific measurement or experiment needed. An incident diagnosis need not choose a replacement
+ topology: distinguish observed timing/counters from the cache-loss hypothesis and name evidence
+ that would refute it. State which checks actually ran; a paper estimate is not demonstrated
+ failure tolerance.
## Primary sources
- [Redis Cluster specification](https://redis.io/docs/latest/operate/oss_and_stack/reference/cluster-spec/)
- [Redis replication](https://redis.io/docs/latest/operate/oss_and_stack/management/replication/)
+ - [Hazelcast 5.6 Near Cache](https://docs.hazelcast.com/hazelcast/5.6/cluster-performance/near-cache) —
+ an example of invalidation propagation and loss reconciliation, not a universal L1 guarantee.
- [Amazon Dynamo paper](https://www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf)
## References
- [Cache topologies](references/topologies.md) — client-side sharded, proxy-fronted,
clustered and fully replicated compared on failure behaviour, operational cost, client
complexity and consistency, with the near-cache layer and a decision table. Read when
choosing or changing a topology, or when a client library's sharding is in question.
- [Node loss and origin protection](references/node-loss-and-origin-protection.md) — the miss
storm computed from real numbers, replication factor as the lever, warming, coalescing,
origin admission control, and the kill-a-node-under-load test with the bound it asserts.
Read before sizing a cache tier, after any incident where the origin saturated, or when
planning a cache upgrade or restart.