performance-testing · diff
v1.0 to v2.0
180 added, 372 removed. Audit A to A.
---
name: performance-testing
description: >-
- Test application performance with k6 load/stress/soak scripts, Lighthouse CI for
- Web Vitals, and performance budgets as CI gates. Covers load profiles, custom metrics,
- bottleneck identification, and Core Web Vitals monitoring.
- Use when: "performance test," "load test," "k6," "Lighthouse," "Web Vitals,"
- "stress test," "performance budget."
+ Test application performance with k6 load/stress/soak/spike scripts and k6 scenarios,
+ Lighthouse CI for Web Vitals, and performance budgets as CI gates. Covers load profiles,
+ custom metrics, bottleneck identification, and Core Web Vitals (LCP, INP, CLS).
+ Use when: "performance test," "load test," "stress test," "soak test," "spike test,"
+ "k6," "k6 scenarios," "Lighthouse," "Web Vitals," "Core Web Vitals," "performance budget."
+ Not for: scheduled production probes — use synthetic-monitoring; pixel-diff regressions —
+ use visual-testing; designing tests from prod telemetry — use observability-driven-testing.
Related: ci-cd-integration, qa-metrics, release-readiness.
license: MIT
metadata:
author: kindlmann
- version: "1.0"
+ version: "2.0"
category: automation
---
<objective>
- Measure, assert, and protect application performance. This skill covers two domains: **load testing** (can the backend handle traffic?) and **web performance** (is the frontend fast for users?). Both use measurable budgets enforced in CI, not subjective "feels fast enough" assessments.
+ Measure, assert, and protect application performance with budgets enforced in CI, not
+ subjective "feels fast enough" assessments. This skill covers two domains: **load testing**
+ (can the backend handle traffic?) and **web performance** (is the frontend fast for users?).
+ A report that says "LCP is 3.2s" is information; a CI gate that fails the build at 2.5s is
+ accountability.
</objective>
- ---
+ ## Quick Route
+ | You need to... | Go to |
+ |----------------|-------|
+ | Test backend capacity / throughput / latency under traffic | **k6 Load Testing** → `references/recipes.md` |
+ | Pick a load shape (constant / ramp / spike / soak) | **Load Profiles** table |
+ | Measure frontend speed for real users (LCP, INP, CLS) | **Web Performance** + **Core Web Vitals** |
+ | Gate page perf in CI | **Lighthouse CI** (gate TBT, not INP — see below) |
+ | A budget is breached and you must find why | **Bottleneck Identification** |
+ | Migrate an existing suite from k6 v1 → v2 | **k6 v2 Migration** callout |
+
## Discovery Questions
- Before writing performance tests, gather context. Check `.agents/qa-project-context.md` first -- if it exists, use it and skip questions already answered there.
+ Check `.agents/qa-project-context.md` first — if it exists, use it and skip questions
+ already answered there.
### What to Measure
-
- - **Web performance or load testing?** Web performance measures user-perceived speed (Core Web Vitals, page load time). Load testing measures server capacity (requests per second, response time under load). Most products need both.
- - **What are the critical user journeys for performance?** Not every endpoint needs load testing. Focus on high-traffic, revenue-critical, or latency-sensitive flows.
- - **Are there existing performance budgets?** If yes, what are the targets? If no, this skill helps establish them.
+ - **Web performance or load testing?** Web performance measures user-perceived speed (Core Web Vitals, page load). Load testing measures server capacity (RPS, latency under load). Most products need both.
+ - **Which user journeys are performance-critical?** Not every endpoint needs load testing. Focus on high-traffic, revenue-critical, or latency-sensitive flows.
+ - **Existing performance budgets?** If yes, what are the targets? If no, this skill helps establish them — measure baseline first.
### Current State
-
- - **What is the current baseline performance?** You need numbers before you can set targets. Measure first, then define budgets.
- - **What broke due to performance in the past?** Slow pages that lost users, endpoints that timed out under load, database queries that locked up.
- - **Is there existing monitoring?** APM tools (Datadog, New Relic), Real User Monitoring (RUM), or synthetic monitoring that provides field data.
+ - **What is the current baseline?** You need numbers before you can set targets. Measure first, then define budgets.
+ - **What broke due to performance before?** Slow pages that lost users, endpoints that timed out under load, queries that locked up — these point at where to focus.
+ - **Existing monitoring?** APM (Datadog, New Relic), RUM, or synthetic probes give you field data to model realistic load.
### Infrastructure
-
- - **Where does load testing run?** Load tests should target staging or a dedicated load-test environment, never production without explicit coordination.
- - **What is the expected traffic pattern?** Steady traffic, daily peaks, seasonal spikes (Black Friday), or event-driven bursts.
- - **Are there rate limits, WAF rules, or auto-scaling that affect test results?** These must be accounted for in test design.
-
- ---
+ - **Where does load testing run?** Target staging or a dedicated load-test environment, never production without explicit operations coordination.
+ - **Expected traffic pattern?** Steady, daily peaks, seasonal spikes (Black Friday), or event-driven bursts — this picks the load profile.
+ - **Rate limits, WAF rules, or auto-scaling in the path?** These distort results and must be accounted for in test design.
## Core Principles
- ### 1. Measure Before Optimizing
-
- Performance intuition is unreliable. Developers often optimize the wrong thing. Measure first, identify the actual bottleneck, then optimize. A profiled 50ms improvement in the right place beats an assumed 500ms improvement in the wrong place.
-
- ### 2. Budgets as CI Gates
-
- Performance budgets are only useful if they are enforced. A budget documented in a wiki but not checked in CI will be violated within weeks. Wire budgets into the pipeline so regressions fail the build.
-
- ### 3. Realistic Load Is More Valuable Than Maximum Stress
-
- A stress test that pushes the system to 10x expected traffic tells you the breaking point. A load test at 1.5x expected traffic tells you whether tomorrow's real users will have a good experience. Both have value, but realistic load tests run more frequently and catch regressions earlier.
-
- ### 4. Core Web Vitals = User-Perceived Performance
+ ### 1. Measure before optimizing
+ Performance intuition is unreliable; developers routinely optimize the wrong thing. Profile first, identify the actual bottleneck, then optimize. A profiled 50ms win in the right place beats an assumed 500ms win in the wrong one.
- Server-side metrics (response time, throughput) matter, but users experience performance through the browser. Core Web Vitals (LCP, INP, CLS) measure what users actually feel. A fast API that renders slowly in the browser is still slow to users.
+ ### 2. Budgets are only real if CI enforces them
+ A budget documented in a wiki but not checked in CI is violated within weeks. Wire budgets into the pipeline as k6 thresholds and Lighthouse assertions so regressions fail the build, not a quarterly review.
- ### 5. Performance Is a Feature
+ ### 3. Realistic load beats maximum stress
+ A stress test to 10x traffic tells you the breaking point. A load test at 1.5x expected traffic tells you whether tomorrow's real users have a good experience. Both have value, but realistic load runs more often and catches regressions earlier.
- Performance does not happen by accident. It requires dedicated test infrastructure, budgets, monitoring, and the same continuous attention as functional correctness. Treat performance regressions with the same urgency as functional bugs.
+ ### 4. Core Web Vitals are what users actually feel
+ Server-side metrics (latency, throughput) matter, but users experience performance through the browser. LCP, INP, and CLS measure perceived speed. A fast API that renders slowly is still slow to users.
- ---
+ ### 5. Performance is a feature
+ It does not happen by accident. It needs dedicated test infrastructure, budgets, and monitoring, and the same continuous attention as functional correctness. Treat a performance regression with the same urgency as a functional bug.
## k6 Load Testing
- k6 is an open-source load testing tool that uses JavaScript/TypeScript for test scripts. It runs from the command line, integrates with CI, and outputs detailed metrics. Current stable: 1.x. **k6 2.0-rc1** shipped April 2026 with breaking changes — see migration callout below.
+ k6 is an open-source load testing tool that uses JavaScript/TypeScript for scripts, runs
+ from the CLI, and integrates with CI. **Current stable: k6 v2.0.0** (final shipped
+ 2026-05). v2 has breaking changes from v1 — see the migration callout below.
- > **k6 v2 migration (2026-04 onward):**
- > - `k6/experimental/websockets` → `k6/websockets` (drop the `experimental/` prefix; module is now stable)
- > - `k6/experimental/redis` removed entirely (use a custom client or wait for stable replacement)
- > - `externally-controlled` executor removed
- > - `options.ext.loadimpact` removed (Grafana Cloud k6 — formerly k6 Cloud / Load Impact — uses `options.cloud` now)
- > - CLI flags removed: `--no-summary`, `--upload-only`, `k6 login`, `k6 pause` (Grafana Cloud commands now require an explicit stack)
- > - Exit code **97** is new: signals a non-threshold cloud-side abort. Wire it into your CI handling.
- > Reference: https://github.com/grafana/k6/releases/tag/v2.0.0-rc1
+ A load test is built from three pieces: a **load profile** (the `stages`/`scenarios`
+ shape), **checks** (per-request assertions), and **thresholds** (pass/fail budgets that
+ set the exit code). Always drive the base URL from `__ENV` so the same script runs against
+ local, staging, and CI.
- ### Basic Load Test Script
+ See `references/recipes.md` for the full basic load test, custom metrics, and scenarios.
+ A minimal threshold block:
```javascript
- // load-tests/api-load.js
- import http from 'k6/http';
- import { check, sleep } from 'k6';
- import { Rate, Trend } from 'k6/metrics';
-
- // Custom metrics
- const errorRate = new Rate('errors');
- const loginDuration = new Trend('login_duration', true);
-
export const options = {
stages: [
- { duration: '1m', target: 20 }, // Ramp up to 20 users over 1 minute
- { duration: '3m', target: 20 }, // Stay at 20 users for 3 minutes
- { duration: '1m', target: 0 }, // Ramp down to 0
+ { duration: '1m', target: 20 },
+ { duration: '3m', target: 20 },
+ { duration: '1m', target: 0 },
],
thresholds: {
- http_req_duration: ['p(95)<500', 'p(99)<1000'], // 95th percentile under 500ms
- http_req_failed: ['rate<0.01'], // Less than 1% failure rate
- errors: ['rate<0.05'], // Custom error rate under 5%
+ http_req_duration: ['p(95)<500', 'p(99)<1000'],
+ http_req_failed: ['rate<0.01'],
},
};
-
- const BASE_URL = __ENV.BASE_URL || 'http://localhost:3000';
-
- export default function () {
- // Login
- const loginStart = Date.now();
- const loginRes = http.post(`${BASE_URL}/api/auth/login`, JSON.stringify({
- email: 'loadtest@example.com',
- password: 'testpassword',
- }), {
- headers: { 'Content-Type': 'application/json' },
- tags: { name: 'login' },
- });
- loginDuration.add(Date.now() - loginStart);
-
- check(loginRes, {
- 'login returns 200': (r) => r.status === 200,
- 'login returns token': (r) => !!r.json('token'),
- }) || errorRate.add(1);
-
- const token = loginRes.json('token');
-
- // Fetch dashboard
- const dashboardRes = http.get(`${BASE_URL}/api/dashboard`, {
- headers: { Authorization: `Bearer ${token}` },
- tags: { name: 'dashboard' },
- });
-
- check(dashboardRes, {
- 'dashboard returns 200': (r) => r.status === 200,
- 'dashboard has widgets': (r) => r.json('widgets') !== undefined,
- }) || errorRate.add(1);
-
- sleep(1); // Simulate user think time (1 second between actions)
- }
```
### Load Profiles
| Profile | Question | Shape | Duration |
|---------|----------|-------|----------|
| **Constant** | Can the system handle normal traffic? | `vus: 50, duration: '10m'` | 10 min |
- | **Ramp-up** | At what point does the system degrade? | Stages: 50 → 100 → 200 → 400 → 800 → 0 | 12 min |
- | **Spike** | Does the system recover from sudden spikes? | 50 → spike to 500 → sustain → drop to 50 | 6 min |
- | **Soak** | Does the system leak resources over time? | Ramp to 100, sustain 4 hours, ramp down | 4+ hours |
-
- ```javascript
- // Spike test example
- export const options = {
- stages: [
- { duration: '1m', target: 50 }, // Normal load
- { duration: '10s', target: 500 }, // Spike to 10x
- { duration: '2m', target: 500 }, // Sustain spike
- { duration: '10s', target: 50 }, // Drop back to normal
- { duration: '2m', target: 50 }, // Recovery period
- ],
- };
- ```
-
- ### Custom Metrics and Tags
-
- k6 provides four metric types: `Counter` (cumulative count), `Rate` (percentage), `Trend` (statistical distribution), `Gauge` (current value). Use tags to filter metrics by endpoint, scenario, or flow.
-
- ```javascript
- import { Counter, Rate, Trend } from 'k6/metrics';
-
- const ordersCreated = new Counter('orders_created');
- const errorRate = new Rate('checkout_errors');
- const checkoutDuration = new Trend('checkout_duration', true);
-
- export default function () {
- const start = Date.now();
- const res = http.post(`${BASE_URL}/api/orders`, orderPayload, {
- headers: { Authorization: `Bearer ${token}` },
- tags: { name: 'create_order', flow: 'checkout' },
- });
- checkoutDuration.add(Date.now() - start);
- res.status === 201 ? ordersCreated.add(1) : errorRate.add(1);
- }
- ```
+ | **Ramp-up (stress)** | At what point does it degrade? | 50 → 100 → 200 → 400 → 800 → 0 | 12 min |
+ | **Spike** | Does it recover from a sudden surge? | 50 → spike 500 → sustain → drop 50 → recover | 6 min |
+ | **Soak** | Does it leak resources over time? | Ramp to 100, sustain 4h, ramp down | 4+ hours |
- ### Scenarios for Multi-Flow Testing
+ A spike test is not done until "does it recover?" is an **assertion, not a comment**. Tag
+ the post-spike window (e.g. `phase:recovery`) and scope a threshold to it so the run fails
+ if p95 stays elevated. See the recovery-detection recipe in `references/recipes.md`.
- Scenarios simulate different user behaviors concurrently with per-scenario thresholds:
+ ### Custom Metrics and Scenarios
- ```javascript
- export const options = {
- scenarios: {
- browse: { executor: 'constant-vus', vus: 100, duration: '10m', exec: 'browseProducts' },
- checkout: { executor: 'ramping-vus', startVUs: 0, stages: [{ duration: '2m', target: 20 }, { duration: '6m', target: 20 }], exec: 'performCheckout' },
- api_heavy: { executor: 'constant-arrival-rate', rate: 50, timeUnit: '1s', duration: '10m', preAllocatedVUs: 100, exec: 'apiHeavyOperations' },
- },
- thresholds: {
- 'http_req_duration{scenario:browse}': ['p(95)<200'],
- 'http_req_duration{scenario:checkout}': ['p(95)<500'],
- },
- };
- ```
+ k6 has four metric types: `Counter` (cumulative count), `Rate` (proportion of non-zero/true
+ values, 0..1), `Trend` (statistical distribution — p50/p95/p99), `Gauge` (latest value). Tag
+ requests with `{ tags: { name: 'endpoint' } }` to filter metrics per endpoint, scenario, or
+ flow.
- Each `exec` function defines a separate user flow. Tag requests with `{ tags: { name: 'endpoint_name' } }` for per-endpoint analysis.
+ **Scenarios** run distinct user flows concurrently, each with its own executor and
+ per-scenario thresholds (`'http_req_duration{scenario:checkout}': ['p(95)<500']`). Use them
+ to model a real mix — browsers + checkout + API-heavy load at once. Full custom-metric and
+ scenario examples are in `references/recipes.md`.
### k6 CI Integration
- ```yaml
- # GitHub Actions
- name: Performance Tests
- on:
- push:
- branches: [main]
- schedule:
- - cron: '0 2 * * 1-5' # Weekday nights
-
- jobs:
- load-test:
- runs-on: ubuntu-latest
- steps:
- - uses: actions/checkout@v4
- - name: Install k6
- run: |
- sudo gpg -k
- sudo gpg --no-default-keyring --keyring /usr/share/keyrings/k6-archive-keyring.gpg --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys C5AD17C747E3415A3642D57D77C6C491D6AC1D68
- echo "deb [signed-by=/usr/share/keyrings/k6-archive-keyring.gpg] https://dl.k6.io/deb stable main" | sudo tee /etc/apt/sources.list.d/k6.list
- sudo apt-get update && sudo apt-get install k6
- - name: Run load tests
- run: k6 run load-tests/api-load.js --out json=results.json
- env:
- BASE_URL: ${{ secrets.STAGING_URL }}
- - name: Upload results
- if: always()
- uses: actions/upload-artifact@v4
- with:
- name: k6-results
- path: results.json
- ```
-
- ---
-
- ## Lighthouse CI
-
- Lighthouse CI automates Google Lighthouse audits and enforces performance budgets in the pipeline.
-
- ```javascript
- // lighthouserc.js
- module.exports = {
- ci: {
- collect: {
- url: ['http://localhost:3000/', 'http://localhost:3000/products', 'http://localhost:3000/checkout'],
- startServerCommand: 'npm run start',
- numberOfRuns: 3,
- settings: { preset: 'desktop' },
- },
- assert: {
- assertions: {
- 'largest-contentful-paint': ['error', { maxNumericValue: 2500 }],
- 'cumulative-layout-shift': ['error', { maxNumericValue: 0.1 }],
- 'total-blocking-time': ['error', { maxNumericValue: 200 }],
- 'categories:performance': ['error', { minScore: 0.9 }],
- 'resource-summary:script:size': ['warn', { maxNumericValue: 300000 }],
- 'resource-summary:total:size': ['warn', { maxNumericValue: 1000000 }],
- 'interactive': ['error', { maxNumericValue: 5000 }],
- },
- },
- upload: { target: 'temporary-public-storage' },
- },
- };
- ```
+ Install k6 with the official **`grafana/setup-k6-action@v1`** — not a hand-rolled apt/gpg
+ keyserver block (brittle, rots, no version pin). k6 exits non-zero when any threshold is
+ breached, so a breached budget fails the job with no extra wiring. The full GitHub Actions
+ workflow (checkout → setup-k6 → run → upload artifact) is in `references/recipes.md`.
- ```yaml
- # CI: npm install -g @lhci/cli && lhci autorun
- - name: Lighthouse CI
- run: npm install -g @lhci/cli && lhci autorun
- env:
- LHCI_GITHUB_APP_TOKEN: ${{ secrets.LHCI_GITHUB_APP_TOKEN }}
- ```
+ > **k6 v1 → v2 migration (v2.0.0 final, 2026-05):**
+ > - `k6/experimental/websockets` → `k6/websockets` (drop the `experimental/` prefix; stable now)
+ > - `k6/experimental/redis` → **`k6/x/redis`** — NOT removed. The import auto-resolves the
+ > `xk6-redis` extension (auto-extension-resolution is on by default; JS usage unchanged).
+ > Do not hand-roll a Redis client.
+ > - `externally-controlled` executor removed
+ > - `options.ext.loadimpact` removed → use `options.cloud` (Grafana Cloud k6, formerly k6 Cloud / Load Impact)
+ > - CLI: `--no-summary` → **`--summary-mode=disabled`**; `--upload-only` → `k6 cloud upload script.js`;
+ > `k6 login`/`pause`/`resume`/`scale`/`status` removed (use `k6 cloud login`, etc.); positional `k6 cloud script.js` removed
+ > - Exit code **97** is new: a non-threshold cloud-side abort. Wire it into CI handling.
+ > Reference: https://grafana.com/docs/k6/latest/get-started/migrating-to-v2/
- For historical tracking, self-host an LHCI server (`lhci server --storage.storageMethod=sql`) to detect gradual degradation and correlate performance changes with commits.
+ ## Web Performance
- LHCI baseline: `@lhci/cli` 0.15.x is current. Lighthouse 12 is the current engine, with INP scoring weights updated. Project activity has slowed; LHCI remains the standard CI surface for Lighthouse but watch upstream for any deprecation signals before adopting in new projects.
+ ### Lighthouse CI
- ### k6 Browser Module
+ Lighthouse CI (`@lhci/cli`) automates Google Lighthouse audits and enforces budgets in the
+ pipeline via `lighthouserc.js` assertions. Current: `@lhci/cli` 0.15.x on the Lighthouse 12.6
+ engine. The project is in maintenance mode (last release ~a year ago) and does not yet
+ support Lighthouse 13 (needs Node 22.19+); it remains the standard CI surface for Lighthouse,
+ but watch upstream before adopting in greenfield projects.
- The `k6/browser` module (built on Playwright) is now stable in k6 1.x. Use it to capture Core Web Vitals (LCP, INP, CLS) under load — bridges traditional protocol-level load testing with full-page performance.
+ The full `lighthouserc.js` (LCP/CLS/TBT/perf-score assertions) and the `lhci autorun` CI
+ step are in `references/recipes.md`.
- ```javascript
- import { browser } from 'k6/browser';
- import { check } from 'k6';
+ ### INP is field-only — gate TBT in the lab
- export const options = {
- scenarios: {
- ui: {
- executor: 'shared-iterations',
- options: { browser: { type: 'chromium' } },
- vus: 5, iterations: 25,
- },
- },
- };
+ This is the single most-misunderstood point in web perf, so be precise:
- export default async function () {
- const page = await browser.newPage();
- try {
- await page.goto(__ENV.BASE_URL || 'http://localhost:3000');
- const lcp = await page.evaluate(() => new Promise((resolve) => {
- new PerformanceObserver((list) => resolve(list.getEntries().pop()?.startTime ?? 0)).observe({ type: 'largest-contentful-paint', buffered: true });
- }));
- check(lcp, { 'LCP < 2500ms': (v) => v < 2500 });
- } finally {
- await page.close();
- }
- }
- ```
+ - **INP (Interaction to Next Paint) is a field-only metric.** A Lighthouse lab audit loads
+ a page with no user interaction, so it **cannot measure or score INP**. Asserting
+ `interaction-to-next-paint` in a standard `lhci autorun` run gates something Lighthouse
+ never produces.
+ - **In the lab, gate Total Blocking Time (TBT)** as the proxy: assert
+ `'total-blocking-time': ['error', { maxNumericValue: 200 }]`. TBT correlates with INP but
+ is not identical (a page can hit 0ms TBT and still fail field INP).
+ - **Track real INP from the field** — Chrome UX Report (CrUX) or your RUM tool. That is the
+ number users actually experience and the one Google ranks on.
+ - **Only if you must measure scripted-interaction latency**, use Lighthouse user-flow /
+ timespan mode with scripted clicks, or `k6/browser` with a `PerformanceObserver` on
+ `event` entries. Treat that as a scripted lab proxy, still not field INP.
- For continuous monitoring of synthetic and real-user perf, see also WebPageTest, Calibre, and SpeedCurve as alternative platforms; **Grafana Cloud k6** is the managed platform formerly known as k6 Cloud / Load Impact.
+ > **FID is gone:** FID was deprecated and `web-vitals` v5+ removed it; INP became a Core Web
+ > Vital in March 2024. Do not assert on FID in any new code.
- > **FID is gone:** `web-vitals` v5+ removed FID entirely; INP replaced it for Lighthouse 12 and Core Web Vitals scoring. Do not assert on FID in any new code.
+ ### Measuring CWV in Playwright
- ---
+ For a per-page lab check inside your existing Playwright suite, use `page.evaluate` with a
+ `PerformanceObserver` to capture LCP and CLS (both observe cleanly on page load), then assert
+ the thresholds. Full test in `references/recipes.md`. For under-load CWV capture, use the
+ `k6/browser` recipe in the same file.
## Core Web Vitals
- The three metrics Google uses to measure user-perceived performance.
-
- ### LCP (Largest Contentful Paint)
-
- Measures loading performance -- when the largest visible content element finishes rendering.
-
- | Rating | Threshold |
- |--------|-----------|
- | Good | <= 2.5s |
- | Needs Improvement | 2.5s - 4.0s |
- | Poor | > 4.0s |
-
- **Common LCP issues and fixes:**
- - Slow server response: optimize backend, add caching, use CDN
- - Render-blocking resources: defer non-critical CSS/JS, inline critical CSS
- - Slow resource load: optimize images (WebP/AVIF), preload LCP image
- - Client-side rendering: use SSR/SSG for LCP content
-
- ### INP (Interaction to Next Paint)
-
- Measures responsiveness -- the delay between user interaction and the next visual update.
-
- | Rating | Threshold |
- |--------|-----------|
- | Good | <= 200ms |
- | Needs Improvement | 200ms - 500ms |
- | Poor | > 500ms |
-
- **Common INP issues and fixes:**
- - Long JavaScript tasks: break up long tasks with `scheduler.yield()` or `requestIdleCallback`
- - Heavy event handlers: debounce/throttle input handlers, move computation to Web Workers
- - Layout thrashing: batch DOM reads/writes, use `requestAnimationFrame`
-
- ### CLS (Cumulative Layout Shift)
-
- Measures visual stability -- how much the page layout shifts during loading.
+ The three metrics Google uses for user-perceived performance.
- | Rating | Threshold |
- |--------|-----------|
- | Good | <= 0.1 |
- | Needs Improvement | 0.1 - 0.25 |
- | Poor | > 0.25 |
+ | Metric | Measures | Good | Needs Improvement | Poor |
+ |--------|----------|------|-------------------|------|
+ | **LCP** (Largest Contentful Paint) | Loading — when the largest element renders | ≤ 2.5s | 2.5s–4.0s | > 4.0s |
+ | **INP** (Interaction to Next Paint) | Responsiveness — interaction → next paint (field-only) | ≤ 200ms | 200ms–500ms | > 500ms |
+ | **CLS** (Cumulative Layout Shift) | Visual stability — unexpected layout movement | ≤ 0.1 | 0.1–0.25 | > 0.25 |
- **Common CLS issues and fixes:**
- - Images without dimensions: always set `width` and `height` attributes
- - Dynamically injected content: reserve space with CSS `aspect-ratio` or `min-height`
- - Web fonts causing FOUT: use `font-display: swap` with `size-adjust`
- - Ads and embeds: reserve space with fixed-size containers
+ **Common fixes:**
+ - **LCP** — slow server (cache/CDN, SSR/SSG the LCP content), render-blocking CSS/JS (defer, inline critical CSS), slow image load (WebP/AVIF, `preload` the LCP image).
+ - **INP** — long JS tasks (`scheduler.yield()`, `requestIdleCallback`), heavy handlers (debounce/throttle, Web Workers), layout thrashing (batch DOM reads/writes via `requestAnimationFrame`).
+ - **CLS** — set `width`/`height` on images, reserve space for injected content (`aspect-ratio`/`min-height`), `font-display: swap` with `size-adjust`, fixed-size containers for ads/embeds.
### Field vs. Lab Data
| | Lab Data | Field Data |
|---|----------|-----------|
- | **Source** | Lighthouse, WebPageTest, Playwright | Chrome UX Report (CrUX), RUM tools |
+ | **Source** | Lighthouse, WebPageTest, Playwright, k6/browser | Chrome UX Report (CrUX), RUM tools |
| **Environment** | Simulated, controlled | Real users, real devices, real networks |
| **Use for** | Debugging, CI gates, pre-deployment | Understanding actual user experience |
- | **Limitation** | Does not reflect real-world variance | Cannot reproduce specific conditions |
-
- Use lab data for CI gates and debugging. Use field data for understanding the real user experience. A page that scores 100 in Lighthouse but has poor CrUX data has a real problem.
-
- ### Measuring in Playwright
-
- Use `page.evaluate` with `PerformanceObserver` to capture LCP and CLS:
-
- ```typescript
- test('checkout page meets Core Web Vitals', async ({ page }) => {
- await page.goto('/checkout');
-
- const lcp = await page.evaluate(() => new Promise<number>((resolve) => {
- new PerformanceObserver((list) => {
- const entries = list.getEntries();
- resolve(entries[entries.length - 1].startTime);
- }).observe({ type: 'largest-contentful-paint', buffered: true });
- setTimeout(() => resolve(-1), 10000);
- }));
- expect(lcp).toBeLessThan(2500);
-
- const cls = await page.evaluate(() => new Promise<number>((resolve) => {
- let score = 0;
- new PerformanceObserver((list) => {
- for (const e of list.getEntries() as any[]) { if (!e.hadRecentInput) score += e.value; }
- }).observe({ type: 'layout-shift', buffered: true });
- setTimeout(() => resolve(score), 3000);
- }));
- expect(cls).toBeLessThan(0.1);
- });
- ```
+ | **Limitation** | No real-world variance; no field INP | Cannot reproduce specific conditions |
- ---
+ Use lab data for CI gates and debugging; use field data to understand the real user
+ experience. A page that scores 100 in Lighthouse but has poor CrUX data has a real problem —
+ and INP only ever shows up in the field column.
## Bottleneck Identification
- When performance tests reveal problems, investigate in this order:
-
- **1. Identify slow endpoints:** Create per-endpoint `Trend` metrics in k6 to see p50/p95/p99 for each API. The slowest endpoint is the first investigation target.
-
- **2. Database queries:** Check for missing indexes (`EXPLAIN`), N+1 query patterns (use JOINs/batch), lock contention on write-heavy tables (optimize transactions), and unbounded result sets (add pagination).
-
- **3. CDN/Caching:** Verify `cache-control` headers on static assets (`public, max-age=31536000, immutable`) and check `x-cache: HIT` rate.
-
- **4. Third-party scripts:** Run Lighthouse with `blockedUrlPatterns` for analytics/chat/tracking scripts, compare scores with and without them to quantify their impact.
+ When a budget is breached, investigate in this order — never guess, never optimize before
+ profiling:
- ---
+ 1. **Identify the slow endpoint.** Per-endpoint `Trend` metrics in k6 give p50/p95/p99 per API. The slowest is the first target.
+ 2. **Database.** Check missing indexes (`EXPLAIN`), N+1 patterns (JOIN/batch), lock contention on write-heavy tables (optimize transactions), unbounded result sets (paginate).
+ 3. **CDN/caching.** Verify `cache-control` on static assets (`public, max-age=31536000, immutable`) and the `x-cache: HIT` rate.
+ 4. **Third-party scripts.** Run Lighthouse with `blockedUrlPatterns` for analytics/chat/tracking; compare scores with and without to quantify the cost.
+ 5. **Correlate with server metrics.** Client-side latency alone hides the cause — check CPU, memory, disk I/O, connection-pool saturation, and query execution time during the run.
## Anti-Patterns
- ### Load Testing Production Without Coordination
-
- Running load tests against production infrastructure without warning the operations team. Load tests can trigger auto-scaling (expensive), rate limiting (test fails), alerts (unnecessary pages), or actual outages (worst case). Always coordinate with operations and test against staging or a dedicated load-test environment.
-
- ### Unrealistic Load Scenarios
-
- Testing with 10,000 concurrent users when the product has 500 daily active users. Or testing with uniform traffic when real traffic has peaks and valleys. Use analytics data to model realistic load profiles. If you do not have analytics, start with 2x your estimated peak and increase from there.
-
- ### Ignoring Server-Side Metrics
+ ### 1. Load testing production without coordination
+ Load tests can trigger auto-scaling (expensive), rate limiting (test fails), alerts (unnecessary pages), or outages. Always coordinate with operations and target staging or a dedicated load-test environment.
- Running k6 and only looking at client-side response times. The server might be at 95% CPU, the database connection pool might be exhausted, or memory might be leaking. Correlate load test results with server metrics (CPU, memory, disk I/O, connection count, query execution time).
+ ### 2. Unrealistic load scenarios
+ Testing 10,000 concurrent users when the product has 500 DAU, or uniform traffic when real traffic has peaks. Model load from analytics; absent that, start at 2x estimated peak and increase.
- ### Performance Testing Only Before Releases
+ ### 3. Ignoring server-side metrics
+ Looking only at k6's client-side response times. The server may be at 95% CPU, the connection pool exhausted, or memory leaking. Correlate load results with server metrics.
- Running load tests once per quarter before a major release. By then, dozens of performance regressions have accumulated and the root causes are impossible to trace. Run performance tests in CI on every merge to main, with budgets that catch regressions immediately.
+ ### 4. Performance testing only before releases
+ A quarterly pre-release load test lets dozens of regressions accumulate with untraceable root causes. Run in CI on every merge to main with budgets that catch regressions immediately.
- ### No Performance Budgets
+ ### 5. No performance budgets
+ "LCP is 3.2s" is information; "LCP must be under 2.5s" is a gate. Define budgets, enforce them as k6 thresholds and Lighthouse assertions, and treat violations as bugs.
- Measuring performance without defining acceptable thresholds. A report that says "LCP is 3.2 seconds" is information. A CI gate that says "LCP must be under 2.5 seconds" is accountability. Define budgets, enforce them in CI, and treat violations as bugs.
+ ### 6. Asserting INP in standard Lighthouse CI
+ Standard `lhci autorun` lab mode cannot produce an INP audit — no user interaction. Gating `interaction-to-next-paint` there gates a number Lighthouse never measures. Gate `total-blocking-time` as the lab proxy and track real INP from CrUX/RUM.
- ### Optimizing Without Profiling
+ ### 7. Optimizing without profiling
+ Spending days on a function that is 2% of response time. Profile, find the actual bottleneck, then optimize. The 200ms query beats the 5ms JS function every time.
- Spending days optimizing a function that contributes 2% of the total response time. Profile first, find the actual bottleneck, then optimize. The database query that takes 200ms is more impactful than the JavaScript function that takes 5ms.
+ ### 8. Testing with empty databases
+ Load testing against 100 seeded rows when production has 10 million. Query performance is radically different at scale. Seed the load-test environment with production-scale (anonymized) data first.
- ### Testing With Empty Databases
+ ## Verification
- Running load tests against a freshly seeded database with 100 records when production has 10 million. Query performance is radically different at scale. Seed the load-test environment with production-scale data (anonymized) before testing.
+ Prove the artifacts actually work before claiming done:
- ---
+ - **k6 script:** `k6 run --summary-mode=disabled load-tests/api-load.js` exits **0** and the
+ end-of-test summary shows every `thresholds` line green (`✓`). A red threshold line means a
+ breached budget and a non-zero exit.
+ - **Spike recovery:** run the spike script and confirm the `{phase:recovery}` threshold
+ appears in the summary and passes — proving recovery is asserted, not just commented.
+ - **Lighthouse CI:** `lhci autorun` exits **0** with all `['error', ...]` assertions passing;
+ a breached LCP/CLS/TBT assertion exits non-zero.
## Done When
- - k6 script covers all target load scenarios: baseline (normal traffic), stress (ramp to breaking point), and soak (sustained load over hours).
- - Performance budgets defined for critical flows and encoded as k6 thresholds (e.g., `p(95)<500`, `http_req_failed rate<0.01`) that fail the CI job when exceeded.
- - Lighthouse CI configured with a `budget.json` that gates merges on LCP, INP, CLS, and total JS/resource size.
+ - k6 scripts cover the target load profiles: baseline (constant), stress (ramp to breaking point), and soak (sustained), each with `__ENV`-driven base URL.
+ - Spike test asserts recovery via a threshold scoped to the post-spike window (e.g. `http_req_duration{phase:recovery}`), not just a `// recovery` comment.
+ - Performance budgets encoded as k6 thresholds (e.g. `p(95)<500`, `http_req_failed rate<0.01`) that fail the CI job when exceeded.
+ - Lighthouse CI `lighthouserc.js` gates merges on `largest-contentful-paint`, `cumulative-layout-shift`, and `total-blocking-time` (the lab proxy for INP) as `['error', ...]` assertions; real INP tracked from CrUX/RUM, not asserted in lab.
- Core Web Vitals baselines documented for each key page (home, checkout, dashboard) with Good/Needs Improvement/Poor classification.
- - Test results include p95 and p99 latency, error rate, and throughput for each scenario, stored as CI artifacts.
+ - Test results include p95 and p99 latency, error rate, and throughput per scenario, stored as CI artifacts.
+ ## Reference Files (in `references/`)
+
+ - **recipes.md** — runnable artifacts: basic k6 load test, spike-with-recovery detection, custom metrics, scenarios, k6 CI workflow (`grafana/setup-k6-action`), `lighthouserc.js`, the `k6/browser` CWV-under-load example, and the Playwright LCP/CLS test.
+
## Related Skills
- - **ci-cd-integration** -- Pipeline configuration for running k6 and Lighthouse CI, scheduling nightly performance runs, and gating deployments on performance budgets.
- - **qa-metrics** -- Performance metrics (LCP, INP, CLS, p95 response time) as part of the broader QA metrics dashboard.
- - **release-readiness** -- Performance benchmarks as part of the go/no-go release checklist.
- - **qa-project-context** -- The project context file captures performance budgets, expected traffic patterns, and critical flows to test.
+ - **ci-cd-integration** — pipeline wiring for k6 and Lighthouse CI, scheduling nightly runs, gating deployments on budgets.
+ - **qa-metrics** — LCP/INP/CLS and p95 latency as part of the broader QA metrics dashboard.
+ - **release-readiness** — performance benchmarks in the go/no-go checklist.
+ - **synthetic-monitoring** — scheduled production CWV/uptime probes *after* release; this skill is pre-release lab gating.
+ - **observability-driven-testing** — when prod telemetry is the *input* to designing new perf tests.
+ - **qa-project-context** — captures performance budgets, traffic patterns, and critical flows to test.