git:20260316.d7ef49c to git:20260316.7fd397a

---

name: api-database-redis

description: Redis in-memory data store patterns with ioredis and node-redis -- caching, sessions, rate limiting, pub/sub, streams, queues, transactions, cluster

---

# Redis Patterns

> **Quick Guide:** Use Redis as an in-memory data store for caching, session management, rate limiting, pub/sub messaging, and job queues. Use **ioredis** (v5.x) as the primary client for its superior TypeScript support, Cluster/Sentinel integration, auto-pipelining, and Lua scripting. Use **node-redis** (v5.x) only when you need Redis Stack modules (JSON, Search, TimeSeries). Always set `maxRetriesPerRequest: null` for BullMQ workers, use separate connections for Pub/Sub subscribers, and define Lua scripts via `defineCommand` for atomic multi-step operations.

---

<critical_requirements>

## CRITICAL: Before Using This Skill

> **All code must follow project conventions in CLAUDE.md** (kebab-case, named exports, import ordering, `import type`, named constants)

**(You MUST use a SEPARATE Redis connection for Pub/Sub subscribers -- a subscribed connection enters a special mode and cannot execute other commands)**

**(You MUST set `maxRetriesPerRequest: null` on any ioredis connection passed to BullMQ -- BullMQ requires infinite retries and will throw if this is not set)**

**(You MUST use Lua scripts (`defineCommand` or `eval`) for any operation requiring atomicity across multiple Redis commands -- separate commands are NOT atomic even in a pipeline)**

**(You MUST handle the `error` event on every Redis client instance -- unhandled errors crash the Node.js process)**

</critical_requirements>

---

---

**Auto-detection:** Redis, ioredis, node-redis, createClient, RedisStore, BullMQ, Queue, Worker, pub/sub, MULTI, EXEC, pipeline, Lua script, defineCommand, xadd, xread, cache-aside, rate limit, session store, connect-redis, Redis.Cluster, Sentinel

**When to use:**

- Caching database queries or API responses (cache-aside, write-through)

- Session storage for Express/Hono/Fastify applications

- Distributed rate limiting (sliding window, token bucket)

- Real-time messaging with Pub/Sub

- Background job processing with BullMQ queues

- Leaderboards, counters, and real-time analytics with sorted sets

- Distributed locks and atomic operations with Lua scripts

**Key patterns covered:**

- ioredis connection setup, configuration, and error handling

- Data structures (strings, hashes, lists, sets, sorted sets, streams)

- Cache-aside and write-through caching with TTL management

- Session storage with connect-redis

- Rate limiting with Lua scripts (sliding window, token bucket)

- Pub/Sub messaging with separate connections

- Redis Streams for persistent message queues

- BullMQ for job queues with retries and scheduling

- Pipelining and transactions (MULTI/EXEC)

- Lua scripting for atomic operations

- Cluster mode and Sentinel for high availability

**When NOT to use:**

- Primary database for relational data (use PostgreSQL/Drizzle)

- Document storage with complex queries (use MongoDB)

- Large binary file storage (use S3/object storage)

- Data that must survive total memory loss without persistence configured

---

<philosophy>

## Philosophy

Redis is an **in-memory data store** used as a cache, message broker, and streaming engine. The core principle: **use Redis for fast, ephemeral, or real-time data -- not as a primary database.**

**Core principles:**

1. **Cache, don't store** -- Redis complements your primary database. Cache frequently accessed data, but always have a source of truth elsewhere.

2. **Atomic operations** -- Use Lua scripts or MULTI/EXEC for operations spanning multiple keys. Individual Redis commands are atomic, but sequences are not.

3. **Separate concerns** -- Use different Redis databases (or key prefixes) for caching, sessions, and queues. Use separate connections for Pub/Sub.

4. **Set TTLs on everything** -- Memory is finite. Every cached key should expire. Use `EX` (seconds) or `PX` (milliseconds) on SET commands.

5. **Fail gracefully** -- Redis is a cache, not a database. If Redis is down, the application should degrade gracefully (bypass cache, use database directly).

</philosophy>

---

<patterns>

## Core Patterns

### Pattern 1: ioredis Connection Setup

```typescript

// ✅ Good Example - Proper ioredis setup with error handling

import Redis from "ioredis";

const RETRY_DELAY_BASE_MS = 50;

const RETRY_DELAY_MAX_MS = 2000;

function createRedisClient(): Redis {

const url = process.env.REDIS_URL;

if (!url) {

throw new Error("REDIS_URL environment variable is required");

}

const client = new Redis(url, {

maxRetriesPerRequest: 3,

retryStrategy(times) {

},

lazyConnect: true,

});

client.on("error", (err) => {

console.error("Redis connection error:", err.message);

});

return client;

}

export { createRedisClient };

```

```typescript

// ❌ Bad Example - No error handling, hardcoded config

import Redis from "ioredis";

const redis = new Redis("redis://localhost:6379");

```

---

```typescript

async function cacheAside<T>(

redis: Redis,

key: string,

fetcher: () => Promise<T>,

): Promise<T> {

if (cached !== null) {

return JSON.parse(cached) as T;

}

const data = await fetcher();

});

return data;

}

```

---

```typescript

}

```

```typescript

// ❌ Bad Example - Non-atomic balance transfer

await redis.decrby("balance:user1", 100);

await redis.incrby("balance:user2", 100);

```

---

```typescript

redis.defineCommand("rateLimit", {

numberOfKeys: 1,

lua: `

local key = KEYS[1]

local limit = tonumber(ARGV[1])

local window = tonumber(ARGV[2])

local now = tonumber(ARGV[3])

redis.call('ZREMRANGEBYSCORE', key, 0, now - window)

local count = redis.call('ZCARD', key)

if count < limit then

redis.call('ZADD', key, now, now .. '-' .. math.random(1000000))

redis.call('EXPIRE', key, window)

end

`,

});

```

---

```typescript

```

```typescript

const redis = new Redis();

await redis.subscribe("channel");

```

---

```typescript

function createBullMQConnection(): Redis {

return new Redis(process.env.REDIS_URL!, {

maxRetriesPerRequest: null, // REQUIRED for BullMQ

});

}

const emailQueue = new Queue<EmailJobData>(QUEUE_NAME, {

connection: createBullMQConnection(),

defaultJobOptions: {

attempts: MAX_RETRY_ATTEMPTS,

},

});

```

```typescript

const connection = new Redis(); // Missing maxRetriesPerRequest: null

const queue = new Queue("emails", { connection });

// BullMQ will throw: "maxRetriesPerRequest must be null"

```

---

```typescript

);

```

</patterns>

---

<performance>

## Performance Optimization

### Auto-Pipelining

ioredis can automatically batch commands issued during the same event loop tick:

```typescript

const redis = new Redis(process.env.REDIS_URL!, {

enableAutoPipelining: true,

});

// These three commands are automatically batched into one pipeline

const [name, email, role] = await Promise.all([

redis.get("user:name"),

redis.get("user:email"),

redis.get("user:role"),

]);

```

```typescript

const TTL_SHORT_SECONDS = 60; // 1 minute -- volatile data

const TTL_MEDIUM_SECONDS = 300; // 5 minutes -- API response cache

const TTL_LONG_SECONDS = 3600; // 1 hour -- user profiles

const TTL_SESSION_SECONDS = 86400; // 24 hours -- sessions

```

### Scanning Instead of KEYS

```typescript

```

</performance>

---

<decision_framework>

## Decision Framework

### Which Redis Client?

```

Which Redis client should I use?

```

### Which Caching Strategy?

```

How should I cache this data?

```

### Which Data Structure?

```

What Redis data structure should I use?

```

### Which Messaging Pattern?

```

How should I implement real-time messaging?

```

### Atomicity Decision

```

Do I need atomicity across multiple commands?

```

</decision_framework>

---

<integration>

## Integration Guide

**Works with:**

- **Drizzle/Prisma** -- Cache database queries using cache-aside pattern; invalidate cache on writes

- **Express/Hono/Fastify** -- Session storage via connect-redis, rate limiting middleware

- **BullMQ** -- Job queues built on Redis (requires ioredis with `maxRetriesPerRequest: null`)

- **Socket.IO** -- Redis adapter for scaling WebSocket connections across multiple servers

- **Docker/Kubernetes** -- Redis containers for development, Redis Cluster for production

**Replaces / Conflicts with:**

- **Database-backed sessions** -- Redis sessions are faster and reduce database load

- **RabbitMQ/Kafka** (partially) -- Redis Streams and BullMQ cover most queue use cases; use dedicated message brokers for complex routing or massive throughput

</integration>

---

<red_flags>

## RED FLAGS

**High Priority Issues:**

- Using the same connection for Pub/Sub subscribe and regular commands -- subscribed connections cannot execute non-pub/sub commands

- Missing `maxRetriesPerRequest: null` on BullMQ connections -- BullMQ throws immediately without this setting

- Using `KEYS` command in production -- blocks the entire Redis server while scanning all keys

- No `error` event handler on Redis client -- unhandled errors crash the Node.js process

- Storing large objects (> 1 MB) in Redis -- degrades performance and wastes memory; store a reference and fetch from object storage

**Medium Priority Issues:**

- Missing TTL on cached keys -- causes unbounded memory growth until Redis runs out of memory

- Using `del` with many keys instead of `unlink` -- `del` blocks Redis; `unlink` frees memory asynchronously

- Not using pipelining for batch operations -- each command is a separate network round-trip

- Serializing/deserializing complex objects without error handling -- malformed JSON in cache crashes on parse

- Sharing a single Redis connection across BullMQ Queue and Worker -- each needs its own connection

**Common Mistakes:**

- Assuming pipeline commands are atomic -- pipelines batch for network efficiency but do not provide atomicity (use MULTI/EXEC or Lua)

- Forgetting that `hgetall` returns an empty object `{}` for non-existent keys (not `null`) -- check `Object.keys(result).length === 0`

- Using `MULTI/EXEC` without `WATCH` for conditional updates -- transactions execute unconditionally unless you WATCH keys first

- Not handling `null` returns from `GET` -- cache misses return `null`, not `undefined`

- Connecting to Redis without TLS in production -- credentials sent in plaintext over the network

**Gotchas & Edge Cases:**

- Redis `HGETALL` returns all values as strings -- numbers stored with `HSET` come back as strings, requiring explicit parsing

- `EXPIRE` resets when a key is overwritten with `SET` -- if you `SET` a key that already has a TTL, the TTL is removed unless you include `EX`/`PX` in the `SET` command

- Pub/Sub messages are fire-and-forget -- if no subscriber is listening when a message is published, it is lost forever (use Streams for persistence)

- Redis Cluster does not support multi-key operations across different hash slots -- use `{hash-tag}` prefix to force related keys to the same slot

- `WATCH` is connection-scoped -- concurrent requests sharing a connection will interfere with each other's WATCH state

- ioredis auto-pipelining does not work with `WATCH`/`MULTI` or blocking commands (`BRPOP`, `BLPOP`, `XREAD BLOCK`)

</red_flags>

---

<critical_reminders>

## CRITICAL REMINDERS

> **All code must follow project conventions in CLAUDE.md** (kebab-case, named exports, import ordering, `import type`, named constants)

**(You MUST use a SEPARATE Redis connection for Pub/Sub subscribers -- a subscribed connection enters a special mode and cannot execute other commands)**

**(You MUST set `maxRetriesPerRequest: null` on any ioredis connection passed to BullMQ -- BullMQ requires infinite retries and will throw if this is not set)**

**(You MUST use Lua scripts (`defineCommand` or `eval`) for any operation requiring atomicity across multiple Redis commands -- separate commands are NOT atomic even in a pipeline)**

**(You MUST handle the `error` event on every Redis client instance -- unhandled errors crash the Node.js process)**

**Failure to follow these rules will cause pub/sub failures, BullMQ connection errors, race conditions, and application crashes.**

</critical_reminders>