structured-logging-lite · diff
git:20260605.a09e45c to git:20260610.1fb8a11
14 added, 445 removed. Audit A to A.
---
name: structured-logging-lite
description: Logging system design guide covering centralized architecture, field standards, and distributed tracing. Use when designing log systems, establishing standards, or debugging production issues.
---
- # Structured Logging System Design
- ## Core Principles
-
- - **Logs are data** — Every log is a queryable, structured data point
- - **Single source of truth** — One logging configuration, used everywhere
- - **Context is king** — Every log must be traceable to its source
- - **Human + Machine readable** — Structured for parsing, clear for debugging
- - **No secrets** — Never log passwords, tokens, or PII without masking
-
- ---
-
- ## System Architecture
-
- ### The Golden Rule
-
- > **Configure once, use everywhere. Never instantiate loggers directly in business code.**
-
- ### Architecture Overview
-
- ```
- ┌─────────────────────────────────────────────────────────────────┐
- │ APPLICATION LAYER │
- │ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
- │ │ Service A│ │ Service B│ │ Service C│ │ Handler D│ │
- │ └────┬─────┘ └────┬─────┘ └────┬─────┘ └────┬─────┘ │
- │ └─────────────┴──────┬──────┴─────────────┘ │
- │ ▼ │
- │ ┌─────────────────────────────────────────────────────────┐ │
- │ │ LOGGING INFRASTRUCTURE │ │
- │ │ Logger Factory | Context Provider | Formatters │ │
- │ └─────────────────────────────────────────────────────────┘ │
- │ ▼ │
- │ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
- │ │ Console │ │ File │ │ Log Agg. │ │ Alerts │ │
- │ └──────────┘ └──────────┘ └──────────┘ └──────────┘ │
- └─────────────────────────────────────────────────────────────────┘
- ```
-
- ### Project Structure
-
- ```
- src/
- ├── lib/logging/ # Centralized logging infrastructure
- │ ├── index.ts # Public API exports
- │ ├── logger.ts # Core logger implementation
- │ ├── context.ts # Request context management
- │ ├── config.ts # Configuration
- │ └── types.ts # Type definitions
- ├── modules/
- │ ├── users/user.service.ts # import { logger } from '@/lib/logging'
- │ └── orders/order.service.ts
- └── main.ts # Initialize logging once at startup
- ```
-
- ### Anti-Patterns vs Correct Patterns
-
- ```typescript
- // ❌ BAD: Creating loggers everywhere
- class UserService {
- private logger = new Logger({ service: 'user' }); // Duplicated config
- }
-
- // ❌ BAD: Direct console in production
- console.log('User created:', userId); // No structure, no context
-
- // ✅ GOOD: Import from central location
- import { logger, getContextLogger } from '@/lib/logging';
-
- class UserService {
- async createUser(data: CreateUserInput) {
- const log = getContextLogger(); // Gets context automatically
- log.info('Creating user', { email: data.email });
- }
- }
- ```
-
- ---
-
- ## Log Record Schema
-
- ### Tier 1: Essential Fields (Always Required)
-
- | Field | Type | Description | Example |
- |-------|------|-------------|---------|
- | `timestamp` | ISO 8601 | When event occurred | `2025-12-16T10:30:00.123Z` |
- | `level` | string | Log severity | `INFO`, `WARN`, `ERROR` |
- | `message` | string | Human-readable description | `User login successful` |
- | `service` | string | Service/application name | `user-auth` |
-
- ### Tier 2: Tracing Fields (Distributed Systems)
-
- | Field | Type | Description |
- |-------|------|-------------|
- | `trace_id` | string | Request chain identifier (32 hex) |
- | `span_id` | string | Current operation ID (16 hex) |
- | `request_id` | string | HTTP request identifier |
-
- ### Tier 3: Context Fields (Debugging)
-
- | Field | Type | Description |
- |-------|------|-------------|
- | `user_id` | string | User identifier (hashed if needed) |
- | `method` | string | Function/method name |
- | `duration_ms` | number | Operation duration |
- | `error_code` | string | Application error code |
- | `error_message` | string | Error description |
-
- ### Tier 4: Environment Fields (Operations)
-
- | Field | Type | Description |
- |-------|------|-------------|
- | `env` | string | Deployment environment |
- | `version` | string | Service version |
- | `host` | string | Server hostname |
- | `region` | string | Cloud region |
-
- ### Example Log Entry
-
- ```json
- {
- "timestamp": "2025-12-16T10:30:00.123Z",
- "level": "INFO",
- "message": "Order created",
- "service": "order-service",
- "trace_id": "7b2e4f1a9c3d8e5b6a1f2c3d4e5f6a7b",
- "span_id": "1a2b3c4d5e6f7890",
- "request_id": "req_abc123",
- "user_id": "usr_456",
- "duration_ms": 145
- }
- ```
-
- ---
-
- ## Log Levels
-
- | Level | When to Use | Production | Alert |
- |-------|-------------|------------|-------|
- | `DEBUG` | Development diagnostics | Off | No |
- | `INFO` | Normal business operations | On | No |
- | `WARN` | Unexpected but recoverable | On | Optional |
- | `ERROR` | Failures requiring attention | On | Yes |
- | `FATAL` | Application cannot continue | On | Immediate |
-
- ### Level Selection Guide
-
- ```
- DEBUG → Variable states, cache misses, detailed flow
- INFO → Business milestones: "User registered", "Order shipped"
- WARN → Degraded service: "DB slow, retrying", "Rate limit at 80%"
- ERROR → Failures: "Payment failed", "Query timeout after 3 retries"
- FATAL → Unrecoverable: "DB connection lost", "Out of memory"
- ```
-
- ---
-
- ## Centralized Configuration
-
- ```typescript
- // lib/logging/config.ts
- export interface LoggingConfig {
- service: string;
- environment: 'development' | 'staging' | 'production';
- level: 'debug' | 'info' | 'warn' | 'error' | 'fatal';
- format: 'json' | 'pretty';
- redactPaths?: string[]; // ['password', 'token', 'secret']
- }
-
- // lib/logging/index.ts
- let initialized = false;
- let globalLogger: Logger;
-
- export function initializeLogging(config: LoggingConfig): void {
- if (initialized) return;
- globalLogger = createLogger(config);
- initialized = true;
- }
-
- export function getLogger(): Logger {
- if (!initialized) throw new Error('Call initializeLogging() at startup');
- return globalLogger;
- }
- ```
-
- ---
-
- ## Library Selection
-
- | Language | Library | Why |
- |----------|---------|-----|
- | Node.js | **Pino** | Fastest, native JSON |
- | Node.js | Winston | Most popular, flexible |
- | Python | **structlog** | Best structured logging |
- | Go | **zerolog** | Zero allocation |
- | Java | Logback + SLF4J | Industry standard |
- | Rust | tracing | Async-aware |
- | C#/.NET | Serilog | Structured logging pioneer |
-
- ---
-
- ## TypeScript Implementation
-
- A complete, copy-paste ready logging setup:
-
- ```typescript
- // lib/logging/types.ts
- export type LogLevel = 'debug' | 'info' | 'warn' | 'error';
-
- export interface LogEntry {
- timestamp: string;
- level: LogLevel;
- message: string;
- service: string;
- trace_id?: string;
- request_id?: string;
- [key: string]: unknown;
- }
-
- export interface LoggingConfig {
- service: string;
- level: LogLevel;
- pretty?: boolean;
- }
- ```
-
- ```typescript
- // lib/logging/logger.ts
- import { LogEntry, LogLevel, LoggingConfig } from './types';
-
- const LEVELS: Record<LogLevel, number> = { debug: 0, info: 1, warn: 2, error: 3 };
-
- class Logger {
- private config: LoggingConfig;
- private context: Record<string, unknown> = {};
-
- constructor(config: LoggingConfig) {
- this.config = config;
- }
-
- private shouldLog(level: LogLevel): boolean {
- return LEVELS[level] >= LEVELS[this.config.level];
- }
-
- private write(level: LogLevel, message: string, data?: Record<string, unknown>) {
- if (!this.shouldLog(level)) return;
-
- const entry: LogEntry = {
- timestamp: new Date().toISOString(),
- level,
- message,
- service: this.config.service,
- ...this.context,
- ...data,
- };
-
- const output = this.config.pretty
- ? `[${entry.timestamp}] ${level.toUpperCase()} ${message} ${JSON.stringify(data || {})}`
- : JSON.stringify(entry);
-
- console.log(output);
- }
-
- debug(message: string, data?: Record<string, unknown>) { this.write('debug', message, data); }
- info(message: string, data?: Record<string, unknown>) { this.write('info', message, data); }
- warn(message: string, data?: Record<string, unknown>) { this.write('warn', message, data); }
- error(message: string, data?: Record<string, unknown>) { this.write('error', message, data); }
-
- child(context: Record<string, unknown>): Logger {
- const child = new Logger(this.config);
- child.context = { ...this.context, ...context };
- return child;
- }
- }
-
- export { Logger };
- ```
-
- ```typescript
- // lib/logging/index.ts
- import { Logger } from './logger';
- import { LoggingConfig } from './types';
-
- let instance: Logger | null = null;
-
- export function initLogging(config: LoggingConfig): void {
- instance = new Logger(config);
- }
-
- export function getLogger(): Logger {
- if (!instance) throw new Error('Call initLogging() first');
- return instance;
- }
-
- // Usage:
- // initLogging({ service: 'my-app', level: 'info', pretty: process.env.NODE_ENV === 'development' });
- // const log = getLogger();
- // log.info('Server started', { port: 3000 });
- ```
-
- ---
-
- ## Best Practices
-
- ### DO ✅
-
- ```typescript
- // Structured, queryable data
- log.info('Order created', {
- order_id: 'ord_123',
- user_id: 'usr_456',
- total_amount: 99.99,
- items_count: 3
- });
-
- // Include correlation IDs
- log.info('Processing payment', { trace_id, span_id, order_id });
-
- // Log at boundaries
- log.info('API request received', { method: 'POST', path: '/orders' });
- log.info('API response sent', { status: 201, duration_ms: 145 });
- ```
-
- ### DON'T ❌
-
- ```typescript
- // String concatenation (not queryable)
- log.info(`User ${userId} created order ${orderId}`);
-
- // Logging sensitive data
- log.info('Login', { password: '...', credit_card: '...' });
-
- // Wrong log levels
- log.error('Cache miss'); // Should be DEBUG
- log.debug('Payment failed'); // Should be ERROR
-
- // Missing context
- log.error('Something went wrong'); // What? Where? Why?
- ```
-
- ---
-
- ## Sensitive Data Handling
-
- ### Never Log
- - Passwords, API keys, tokens, secrets
- - Full credit card numbers, SSN
- - Private keys
-
- ### Always Mask
-
- | Data Type | Pattern | Example |
- |-----------|---------|---------|
- | Email | Partial | `j***n@example.com` |
- | Phone | Last 4 | `****1234` |
- | Credit Card | Last 4 | `****4242` |
- | IP Address | Partial | `192.168.***.***` |
-
- ```typescript
- export const mask = {
- email: (e: string) => `${e[0]}***${e.split('@')[0].slice(-1)}@${e.split('@')[1]}`,
- phone: (p: string) => `****${p.slice(-4)}`,
- card: (c: string) => `****${c.slice(-4)}`,
- };
- ```
-
- ---
-
- ## Field Naming Conventions
-
- ```yaml
- # Identifiers
- user_id, order_id, product_id, session_id, request_id
-
- # Tracing
- trace_id, span_id, parent_span_id
-
- # Time
- timestamp, duration_ms, started_at, ended_at
-
- # Status
- status, outcome, error_code, error_message
-
- # HTTP
- http_method, http_path, http_status
-
- # Database
- db_system, db_name, db_operation
-
- # Environment
- env, version, host, region, instance_id
- ```
-
- ---
-
- ## Checklist
+ # Structured Logging Lite
- ```markdown
- ## Architecture
- - [ ] Centralized logging module (lib/logging/)
- - [ ] Single configuration source
- - [ ] No direct logger instantiation in business code
+ Use this skill to design practical structured logging schemas, levels, central configuration, masking rules, and operational checklists.
- ## Schema
- - [ ] Tier 1 fields (timestamp, level, message, service)
- - [ ] Tier 2 tracing (trace_id, span_id, request_id)
- - [ ] Field naming convention documented
- - [ ] Sensitive data masking rules
+ ## Routing
- ## Standards
- - [ ] Log levels used correctly
- - [ ] All logs are structured JSON
- - [ ] No sensitive data in logs
- - [ ] Consistent field names across services
- - [ ] No console.log in production
+ - Start here to confirm scope, safety boundaries, and the minimum workflow.
+ - Load `references/full-guide.md` before executing detailed commands, applying templates, or relying on examples from this skill.
+ - Keep the work limited to the user-requested target; do not add adjacent improvements unless the user asks.
- ## Operations
- - [ ] Log aggregation configured
- - [ ] Alerts for ERROR/FATAL levels
- - [ ] Retention policy defined
- ```
+ ## Quick Workflow
- ---
+ 1. Confirm the target, available context, constraints, and done-when signal.
+ 2. Read the relevant section of `references/full-guide.md` for task-specific details.
+ 3. Execute the smallest reversible action that satisfies the request.
+ 4. Verify that examples include required fields, level choices, sensitive-data handling, and integration points for the target stack.
+ 5. Report changed files, commands run, evidence collected, and any remaining risk.
- ## Quick Reference
+ ## References
- ```
- ┌─────────────────────────────────────────────────────────────┐
- │ STRUCTURED LOG ENTRY │
- ├─────────────────────────────────────────────────────────────┤
- │ Tier 1 (Required) │
- │ ├── timestamp → ISO 8601 format │
- │ ├── level → DEBUG|INFO|WARN|ERROR|FATAL │
- │ ├── message → Human readable description │
- │ └── service → Service name │
- ├─────────────────────────────────────────────────────────────┤
- │ Tier 2 (Tracing) │
- │ ├── trace_id → Request chain ID │
- │ ├── span_id → Current operation ID │
- │ └── request_id → HTTP request ID │
- ├─────────────────────────────────────────────────────────────┤
- │ Tier 3 (Context) │
- │ ├── user_id → Who triggered this │
- │ ├── method → Function/method name │
- │ ├── duration_ms → How long it took │
- │ └── error_* → Error details (if applicable) │
- ├─────────────────────────────────────────────────────────────┤
- │ Tier 4 (Environment) │
- │ ├── env → production|staging|dev │
- │ ├── version → Service version │
- │ └── host/region → Where it's running │
- └─────────────────────────────────────────────────────────────┘
- ```
+ - `references/full-guide.md` - complete command patterns, examples, checklists, and edge cases.