ci-cd · diff

git:20260718.b4632b5 to git:20260722.125074b

312 added, 361 removed. Audit A to A.

What this update changes in the checks

No check changed: every fixed check gives the same answer before and after this update.

A tripped check is a label, not a verdict: read the matched text and the lines below. This week's updates that newly trip a check.

---
name: ci-cd
description: >
Load this skill when configuring or reviewing CI/CD pipelines, GitHub Actions
workflows, or automated testing setups. Ensures accessibility regressions are
caught before code reaches production by enforcing quality gates, structured
- reporting, and a zero-debt strategy across all pages and user preferences.
+ reporting, and layered automated + manual testing across all pages and user
+ preferences.
---
# CI/CD Accessibility Skill
> **Canonical source**: `examples/CI_CD_ACCESSIBILITY_BEST_PRACTICES.md` in `mgifford/ACCESSIBILITY.md`
> This skill is derived from that file. When in doubt, the example is authoritative.
+ > The canonical guide states synchronization with this skill is **not automatic**
+ > — re-check this file against canonical periodically.
Apply these rules when adding, reviewing, or maintaining CI/CD accessibility checks.
---
## Core Mandate
- Every CI/CD pipeline must prevent accessibility regressions from reaching production.
- Automated checks are the baseline, not the ceiling — combine rule-based scanning with
- accessibility tree testing and, where practical, virtual screen reader testing.
-
- **Zero-Debt strategy:** target 100 % Lighthouse Accessibility and Performance scores
- on all pages across all devices and user preferences.
-
- ## The Strategy: Local-First and AI-Aligned
-
- Automation is the baseline, not the ceiling. Prefer this stack:
+ CI can prevent known accessibility regressions, preserve tested semantics, and
+ produce evidence for human review. **It cannot establish WCAG conformance by
+ itself.** Automated rules, scores, snapshots, and AI output are inputs to an
+ accessibility evaluation, not substitutes for one. This guide targets WCAG 2.2
+ AA for in-scope content — that target is not itself a conformance claim.
- * **Lighthouse CI** for high-level quality gates (100 % targets).
- * **Playwright + axe-core** for dynamic interactions, mobile states, and light/dark coverage.
- * **Local audits first** to reduce CI noise and shorten feedback loops.
- * **Structured outputs (JSON artifacts)** so findings are AI-ready for triage and remediation workflows.
+ Principles: test complete user tasks and relevant states, not only page loads;
+ run fast deterministic checks locally and repeat in CI; block known
+ regressions using explicit reviewable rules; keep accessibility, performance,
+ security, and functional signals distinct; test the final rendered/sanitized/
+ optimized output; keep humans responsible for severity, alternatives,
+ exceptions, and release decisions; give workflows and agents the least
+ authority needed for their task.
---
## Severity Scale (this skill)
| Level | Meaning |
| --- | --- |
- | **Critical** | Blocks task completion entirely for one or more disability groups |
- | **Serious** | Significantly impairs access; workaround unreasonable to expect |
- | **Moderate** | Creates friction; workaround exists and is not too burdensome |
- | **Minor** | Best-practice gap; marginal impact on access |
-
- ---
-
- ## Security Considerations
-
- This skill references GitHub Actions that fetch and execute remote code. This
- creates a **supply chain risk** (Snyk W012: Potentially malicious external URL).
-
- ### The Risk
-
- GitHub Actions execute code from external repositories:
- - `github/accessibility-scanner@v3` — runs accessibility scans against your site
- - `AGENT_REMEDIATION_WORKFLOW.yml` — runs a Copilot coding agent that applies fixes
- - Any pinned action version could be compromised upstream
-
- ### Mitigations
-
- When implementing these workflows:
-
- 1. **Pin to commit SHAs, not tags** — Tags can be moved; commit SHAs cannot:
- ```yaml
- # Instead of:
- uses: actions/checkout@v4
-
- # Use (replace with actual SHA from action's release page):
- uses: actions/checkout@<commit-sha> # e.g., v4.1.1
- ```
+ | **Critical** | A core or safety-critical task cannot be completed and no reasonable accessible alternative exists |
+ | **Serious** | A major task is blocked or requires an unsafe, unreliable, or highly burdensome workaround |
+ | **Moderate** | A task remains possible but creates substantial difficulty, confusion, delay, or loss of information |
+ | **Minor** | Impact is limited and the task remains understandable and operable |
- 2. **Review action sources** — Only use actions from trusted publishers:
- - `actions/checkout`, `actions/setup-node`, `actions/upload-artifact` (GitHub official)
- - `github/accessibility-scanner` (GitHub official)
- - `dequelabs/axe-core-npm` (axe-core maintainers)
+ **Tool labels (axe `critical`/`serious`/`moderate`/`minor`, Lighthouse scores)
+ are triage inputs, not project severity** — map them into this taxonomy only
+ after evaluating actual user impact on the affected task.
- 3. **Audit before copying workflows** — Review `AGENT_REMEDIATION_WORKFLOW.yml`
- before copying to your repo. Understand that it grants Copilot write access
- to create PRs.
+ ---
- 4. **Use `permissions:` to limit blast radius** — Always declare minimum permissions:
- ```yaml
- permissions:
- contents: read
- issues: write # only if creating issues
- ```
+ ## A Layered Testing Model
- 5. **Review Dependabot PRs** — When Dependabot proposes action updates, review
- the changelog before approving.
+ No single tool covers accessibility requirements. Choose layers by the
+ affected task and the likelihood/consequence of a regression:
- ### Acceptance
+ | Layer | Useful for | Does not establish |
+ | --- | --- | --- |
+ | Static analysis / lint | Fast feedback on detectable source patterns | Runtime behavior or final output |
+ | Unit/component tests | Stable component contracts (names, roles, states) | Whole-page reading and focus order |
+ | Browser rule scans (axe-core) | Detectable rules in rendered states | Complete WCAG coverage or usability |
+ | Semantic assertions (role queries, ARIA snapshots) | Accessibility-tree structure and changes | Exact screen-reader speech |
+ | Visual checks | Theme/layout regressions, focus indicators | Meaning, keyboard access, or text alternatives |
+ | End-to-end tests | Operability, focus, state changes, errors | AT interoperability by itself |
+ | Manual and AT testing | Quality, sequence, interaction, compatibility | Universal behavior across all configurations |
- This pattern is acceptable because:
- - CI/CD automation requires executing external code by design
- - Pinning to SHAs ensures you control exactly which code runs
- - The actions referenced are from established, trusted publishers
- - Repository permissions limit what any action can do
+ Use `skills/manual-testing/SKILL.md` for checks automation cannot complete.
---
- ## Critical: Lighthouse CI Quality Gate
+ ## Critical: Define the CI Contract Before Choosing Tools
- Enforce a strict score threshold. A drop to 99 % accessibility or performance **must
- fail the build**.
+ For each check, document: the user task/page/component/states covered; the
+ command and config file; tool/browser/runtime/rule-set versions; whether the
+ result warns, blocks, or only produces an artifact; the baseline/expected
+ result; the owner and response process for failures; known false positives
+ and untested requirements; and evidence retained for review.
- **`.lighthouserc.js` (strict gate — use once baseline is clean):**
- ```javascript
- module.exports = {
- ci: {
- collect: {
- staticDistDir: './_site',
- numberOfRuns: 1,
- settings: { emulatedFormFactor: 'mobile' },
- },
- assert: {
- assertions: {
- 'categories:accessibility': ['error', { minScore: 1 }],
- 'categories:performance': ['error', { minScore: 1 }],
- },
- },
- },
- };
- ```
+ Do not describe a check as a merge gate unless branch protection actually
+ requires it. Do not describe a warning as a failure.
- **`.lighthouserc.json` (warn-first — use while resolving existing issues):**
+ **Scores are not conformance percentages.** A Lighthouse accessibility score
+ of 100 does not mean 100% WCAG conformance, and 90 does not mean a page is 90%
+ accessible — it measures the weighted audits Lighthouse ran in one
+ configuration. Use individual audit results to investigate regressions. A
+ score threshold can be an additional signal but should not be the only gate;
+ also fail on specific rules/tasks the team has decided are blocking. Keep
+ performance budgets separate — a performance score must not raise or lower
+ accessibility severity.
+
```json
{
"ci": {
"collect": { "staticDistDir": "./_site", "numberOfRuns": 1 },
"assert": {
- "assertions": {
- "categories:accessibility": ["warn", { "minScore": 0.9 }]
- }
+ "assertions": { "categories:accessibility": ["warn", { "minScore": 0.9 }] }
},
"upload": { "target": "filesystem", "outputDir": ".lighthouseci" }
}
}
```
- > Start with `"warn"` + `minScore: 0.9`, then tighten to `"error"` + `minScore: 1`
- > once the existing baseline is clean.
+ This warns; it does not block. Raising the threshold (or moving to `"error"`)
+ can be useful after investigation, but the threshold still does not establish
+ conformance.
---
- ## Critical: axe-core on Every PR
-
- Run axe-core via Playwright on every pull request to catch WCAG violations in
- dynamic content (menus, modals, theme variants).
-
- Run the same accessibility checks in both light and dark color schemes. A PR is
- not fully covered unless the test suite explicitly emulates each scheme and
- fails on violations in either mode. If you use visual regression tests, keep the
- baselines separate for light and dark mode.
-
- #### Dual light/dark mode coverage
+ ## Critical: axe-core on Every PR, Across States and Preferences
- Use one test body and run it twice, once for each color scheme:
+ Scan meaningful states after the page is ready AND after relevant interaction
+ — do not scan only the initial route if users must open a menu, submit a
+ form, or complete another action.
```typescript
- import { test, expect } from '@playwright/test';
import AxeBuilder from '@axe-core/playwright';
+ import { expect, test } from '@playwright/test';
- for (const colorScheme of ['light', 'dark'] as const) {
- test(`A11y: home page in ${colorScheme} mode`, async ({ page }) => {
+ const colorSchemes = ['light', 'dark'] as const;
+
+ for (const colorScheme of colorSchemes) {
+ test(`home page in ${colorScheme} mode`, async ({ page }) => {
await page.emulateMedia({ colorScheme });
await page.goto('/');
+ await expect(page.getByRole('main')).toBeVisible();
const results = await new AxeBuilder({ page })
.withTags(['wcag2a', 'wcag2aa', 'wcag21aa', 'wcag22aa'])
.analyze();
-
expect(results.violations).toEqual([]);
});
}
+
+ test('navigation is tested while expanded', async ({ page }) => {
+ await page.goto('/');
+ await page.getByRole('button', { name: 'Menu' }).click();
+ await expect(page.getByRole('navigation', { name: 'Main' })).toBeVisible();
+
+ const results = await new AxeBuilder({ page }).analyze();
+ expect(results.violations).toEqual([]);
+ });
```
- If the app has visual regression tests, keep separate baselines for each
- scheme. That makes theme-specific failures obvious instead of hiding them in one
- shared snapshot.
+ A clean scan means no violations were found in the tested scope — it does not
+ mean no barriers exist. **Missing axe-core checks on PRs is Critical**;
+ missing a color-scheme pass is also a coverage gap since theme-specific
+ regressions often only appear in the untested scheme.
+ Cover applicable combinations without an unnecessarily expensive matrix:
+ desktop and narrow reflow viewports; system/light/dark theme; default and
+ increased contrast; forced colors; reduced motion; default/hover/focus/active/
+ disabled/loading/success/error states; signed-in/signed-out/timeout/
+ permission states. Use pairwise or risk-based coverage for routine changes;
+ test every combination for high-risk components (authentication, payment,
+ safety, core navigation).
+
+ ---
+
+ ## Serious: Semantic Assertions and ARIA Snapshots
+
+ Role and accessible-name queries provide useful contracts:
+
```typescript
- // tests/a11y.spec.ts
- import { test, expect } from '@playwright/test';
- import AxeBuilder from '@axe-core/playwright';
+ test('contact form exposes its controls and error', async ({ page }) => {
+ await page.goto('/contact');
+ const email = page.getByRole('textbox', { name: 'Email address' });
+ await expect(email).toBeVisible();
+ await page.getByRole('button', { name: 'Send message' }).click();
+ await expect(page.getByText('Enter an email address')).toBeVisible();
+ await expect(email).toHaveAttribute('aria-invalid', 'true');
+ });
- const themes = ['light', 'dark'];
+ test('main navigation retains its semantic structure', async ({ page }) => {
+ await page.goto('/');
+ await expect(page.getByRole('navigation', { name: 'Main' }))
+ .toMatchAriaSnapshot(`
+ - navigation "Main":
+ - list:
+ - listitem:
+ - link "Home"
+ `);
+ });
+ ```
- for (const theme of themes) {
- test(`A11y: Desktop & Mobile in ${theme} mode`, async ({ page }) => {
- await page.emulateMedia({ colorScheme: theme as 'light' | 'dark' });
- await page.goto('/');
+ Review snapshot changes; do not update them mechanically. An ARIA snapshot
+ represents the accessibility tree, not a transcript of what every screen
+ reader will announce — speech output also depends on the AT product, browser,
+ OS, settings, mode, and user action.
- const menuBtn = page.locator('#main-menu-toggle');
- if (await menuBtn.isVisible()) await menuBtn.click();
+ **Guidepup virtual screen reader** (unit-level, exact spoken output):
- const results = await new AxeBuilder({ page })
- .withTags(['wcag2a', 'wcag2aa', 'wcag21aa', 'wcag22aa'])
- .analyze();
+ ```typescript
+ import { virtual } from '@guidepup/virtual-screen-reader';
- expect(results.violations).toEqual([]);
- });
- }
+ it('announces the dialog title and action buttons', async () => {
+ document.body.innerHTML = `
+ <dialog open aria-labelledby="dlg-title">
+ <h2 id="dlg-title">Confirm deletion</h2>
+ <button>Delete</button>
+ </dialog>`;
+ await virtual.start({ container: document.body });
+ const spoken = await virtual.spokenPhraseLog();
+ expect(spoken).toContain('Confirm deletion');
+ await virtual.stop();
+ });
```
- **Missing axe-core checks on PRs is Critical** — dynamic violations are invisible to
- Lighthouse and reach production silently. Missing one of the color-scheme passes
- is also a coverage gap, because theme-specific regressions often only appear in
- the scheme that was not exercised.
+ ---
+ ## Serious: Visual and Preference Regression Tests
+
+ Visual tests can find clipped text, missing focus indicators, theme-specific
+ contrast changes, and reflow regressions. Keep separate reviewed baselines for
+ materially different themes and forced-colors behavior. Do not use screenshot
+ equality as the only contrast test — anti-aliasing, transparency, gradients,
+ and rendering differences make pixel comparison unreliable; confirm computed
+ colors and visually inspect where contrast is critical. Emulate reduced
+ motion for deterministic tests, but never remove an animation from the
+ product merely to make a screenshot pass.
+
---
- ## Serious: GitHub Actions Workflows
+ ## Critical: CI Workflow Security
- ### A. Lighthouse CI on every PR and push to `main`
+ Apply these controls to accessibility workflows the same as build/deploy workflows:
+ * **Pin third-party actions to reviewed full commit SHAs, not tags** — tags can
+ move; commit SHAs cannot
+ * **Declare minimum `GITHUB_TOKEN` permissions explicitly** at workflow or job level
+ * **Set `persist-credentials: false`** on checkout when later steps don't need Git access
+ * **Avoid `pull_request_target`** for workflows that execute untrusted PR code
+ * Pass untrusted values through environment variables or files and quote shell
+ expansions — never interpolate them into executable script text
+ * Do not expose secrets to forked or untrusted code
+ * Restrict network access and tools for AI/transformation steps where practical
+ * Add timeouts, concurrency controls, and bounded crawl limits
+ * Treat uploaded HTML, JSON, screenshots, traces, and logs as potentially sensitive
+ * Keep artifact retention no longer than the review process requires
+ * Use environments and required reviewers for any workflow that can deploy or
+ modify external systems
+ * Do not claim a workflow "has no access to secrets" or "cannot merge" unless
+ its token permissions, event context, tools, and platform controls actually
+ demonstrate that
+
```yaml
- # .github/workflows/lighthouse.yml
- name: Lighthouse CI
+ name: Accessibility checks
on:
pull_request:
push:
branches: [main]
workflow_dispatch:
permissions:
contents: read
+ concurrency:
+ group: accessibility-${{ github.workflow }}-${{ github.ref }}
+ cancel-in-progress: true
+
jobs:
- lighthouse:
+ accessibility:
runs-on: ubuntu-latest
+ timeout-minutes: 20
steps:
- - uses: actions/checkout@v4
- - uses: ruby/setup-ruby@v1
- with: { ruby-version: "3.2", bundler-cache: true }
- - run: bundle exec jekyll build
- - uses: actions/setup-node@v4
- with: { node-version: "22" }
- - run: npm install -g @lhci/cli
- - run: lhci autorun
+ - name: Check out repository
+ uses: actions/checkout@<reviewed-full-commit-sha>
+ with:
+ persist-credentials: false
+ - name: Set up Node.js
+ uses: actions/setup-node@<reviewed-full-commit-sha>
+ with:
+ node-version-file: .nvmrc
+ cache: npm
+ - run: npm ci
+ - run: npx playwright install --with-deps chromium
+ - run: npm run build
+ - run: npm run test:a11y:ci
+ - name: Upload debugging evidence after failure
+ if: failure()
+ uses: actions/upload-artifact@<reviewed-full-commit-sha>
+ with:
+ name: accessibility-test-evidence-${{ github.run_id }}
+ path: test-results
+ retention-days: 7
```
- ### B. Scheduled accessibility scan with alert-fatigue guard
+ Before enabling: confirm actions/inputs exist in current official docs;
+ resolve each action release to its full commit SHA; review transitive action
+ code and release notes; validate fork behavior so untrusted PRs get no
+ privileged secrets; verify the build path matches production; run the
+ workflow in a test branch first.
- Run monthly; skip the scan when open `accessibility` issues already exist so
- developers are not flooded with duplicate noise.
+ ---
+ ## Serious: Scheduled Scanning and the GitHub Accessibility Scanner
+
+ Scheduled scans find drift, dependency changes, and pages missed by PR tests.
+ **Continue scanning even when known issues remain** — pausing every scan
+ because one issue is open can hide new regressions; prefer deduplication,
+ stable fingerprints, and clear ownership over an alert-fatigue pause.
+
+ For each scheduled scan, define: authorized target URLs and crawl boundaries;
+ public vs. authenticated states; frequency/timeout/resource limits; issue
+ deduplication and closure behavior; retention and redaction of reports;
+ triage responsibility; and behavior when the scanner/target is unavailable.
+ Never place authentication secrets, session data, or sensitive content in
+ logs, issue bodies, artifacts, or AI prompts.
+
+ The **GitHub Accessibility Scanner** (`github/accessibility-scanner`) is a
+ public-preview action whose interfaces, permissions, and Copilot integration
+ may change — review its current README, `action.yml`, and security guidance
+ immediately before adoption. It requires a token and documents broader
+ permissions than a read-only scan; treat that credential as privileged, use a
+ dedicated fine-grained credential, and do not assume the default
+ `GITHUB_TOKEN` is sufficient. If also using a separate remediation workflow,
+ configure the scanner to skip Copilot assignment so two remediation systems
+ don't act on the same finding.
+
```yaml
# .github/workflows/accessibility-scan.yml
name: Accessibility Scan (Scheduled)
-
on:
schedule:
- - cron: "0 0 1 * *" # first day of every month
+ - cron: "0 0 1 * *"
workflow_dispatch:
-
permissions:
contents: write
issues: write
pull-requests: write
-
jobs:
accessibility-scanner:
runs-on: ubuntu-latest
steps:
- - uses: actions/checkout@v4
-
- - name: Check for existing open accessibility issues
- id: check_issues
- env:
- GH_TOKEN: ${{ secrets.GH_TOKEN || secrets.GITHUB_TOKEN }}
- run: |
- COUNT=$(gh issue list --label "accessibility" --state open --json number --jq '. | length')
- echo "count=$COUNT" >> $GITHUB_OUTPUT
-
+ - uses: actions/checkout@<reviewed-full-commit-sha>
+ with: { persist-credentials: false }
- name: Run GitHub Accessibility Scanner
- if: steps.check_issues.outputs.count == '0'
- uses: github/accessibility-scanner@v3
+ uses: github/accessibility-scanner@<reviewed-full-commit-sha>
with:
- urls: ${{ vars.ACCESSIBILITY_SCAN_URL || format('https://{0}.github.io/{1}/', github.repository_owner, github.event.repository.name) }}
+ urls: ${{ vars.ACCESSIBILITY_SCAN_URL }}
repository: ${{ github.repository }}
- token: ${{ secrets.GH_TOKEN || secrets.GITHUB_TOKEN }}
- cache_key: accessibility-scan-results
- ```
-
- > Set the `ACCESSIBILITY_SCAN_URL` repository variable to override the default
- > GitHub Pages URL. Multiple URLs can be provided as a newline-separated list.
-
- ### C. Full deep-crawl for AI-ready audit (manual trigger)
-
- ```yaml
- name: Deep Site Audit
- on: workflow_dispatch
-
- jobs:
- crawl:
- runs-on: ubuntu-latest
- steps:
- - uses: actions/checkout@v4
- - run: npx playwright test --reporter=json > audit-report.json
- - uses: actions/upload-artifact@v4
- with:
- name: a11y-json-report
- path: audit-report.json
+ token: ${{ secrets.GH_TOKEN }}
```
---
- ## Serious: Accessibility Tree Testing
-
- Automated WCAG rule checks verify markup compliance; they cannot verify what a
- screen reader **actually announces**. Add accessibility tree tests for complex
- components: SVG diagrams, custom widgets, live regions, navigation landmarks.
-
- ### Playwright aria snapshots (Playwright ≥ v1.46)
-
- Assert the exact accessible name, role, and structure that assistive technologies
- consume — distinct from axe-core rule checks.
-
- ```typescript
- // tests/a11y-tree.spec.ts
- import { test, expect } from '@playwright/test';
-
- test('main navigation is correctly announced', async ({ page }) => {
- await page.goto('/');
- await expect(page.locator('nav[aria-label="Main navigation"]'))
- .toMatchAriaSnapshot(`
- - navigation "Main navigation":
- - list:
- - listitem:
- - link "Home"
- - listitem:
- - link "About"
- `);
- });
-
- test('SVG diagram is exposed as a labelled image', async ({ page }) => {
- await page.goto('/diagrams');
- await expect(page.locator('svg[role="img"]').first())
- .toMatchAriaSnapshot(`
- - img "User Authentication Flowchart":
- `);
- });
- ```
-
- Generate baseline snapshots once with `--update-snapshots`; treat subsequent
- diffs the same as visual regression diffs.
-
- ### Semantic role queries (any Playwright version)
-
- ```typescript
- test('form controls have meaningful accessible names', async ({ page }) => {
- await page.goto('/contact');
- await expect(page.getByRole('textbox', { name: 'Email address' })).toBeVisible();
- await expect(page.getByRole('button', { name: 'Send message' })).toBeEnabled();
- });
- ```
-
- `getByRole()` fails immediately when accessible names are missing or wrong —
- earlier feedback than a manual screen reader audit.
+ ## Serious: AI-Assisted Remediation — Treat All Scan Input as Untrusted
- ### Guidepup virtual screen reader (unit-level)
+ An AI-generated patch is a **proposal, not evidence a barrier is fixed**.
+ Issues, comments, labels, repository content, HTML, SVG, URLs, and scan
+ output are all **untrusted input**.
- For asserting exact spoken output without a real screen reader installed:
+ Required review sequence:
- ```typescript
- import { virtual } from '@guidepup/virtual-screen-reader';
+ 1. A scanner or person records a reproducible finding
+ 2. A human triages the affected task, impact, confidence, and scope
+ 3. A maintainer **manually** triggers the remediation workflow with the issue number
+ 4. Copilot reads the issue as untrusted file data in an isolated copy — without
+ the repository's `.git` directory
+ 5. The workflow uploads a patch, changed-file list, and report (does not commit/push/PR/merge itself)
+ 6. A human reviews and applies an acceptable patch in a normal branch
+ 7. Relevant automated, manual, and AT checks are rerun on the final implementation
- it('announces the dialog title and action buttons', async () => {
- document.body.innerHTML = `
- <dialog open aria-labelledby="dlg-title">
- <h2 id="dlg-title">Confirm deletion</h2>
- <button>Delete</button>
- <button>Cancel</button>
- </dialog>
- `;
- await virtual.start({ container: document.body });
- const spoken = await virtual.spokenPhraseLog();
- expect(spoken).toContain('Confirm deletion');
- expect(spoken).toContain('Delete, button');
- await virtual.stop();
- });
- ```
+ A safe reference remediation workflow: is manual-only and requires explicit
+ confirmation; gives the workflow token **read-only** content and issue
+ permissions; keeps issue content out of shell commands, workflow expressions,
+ and static prompts; limits the agent to read/write tools in an isolated
+ working copy; **denies shell, URL, and memory tools**; blocks changes to
+ protected policy/workflow paths; does not commit, push, open a PR, close an
+ issue, or deploy; retains a review artifact for a limited period.
- GitHub Actions setup for Guidepup:
- ```yaml
- - uses: guidepup/setup-action@v2
- - run: npx jest tests/sr.test.ts
- ```
+ **Do not** trigger privileged AI remediation directly from an issue label, or
+ paste an issue body into a shell command, `GITHUB_OUTPUT`, workflow
+ expression, or static prompt — these are the classic script-injection paths
+ for untrusted GitHub content.
---
- ## Moderate: Local-First Developer Workflow
+ ## Moderate: Local-First, Reproducible Setup
- Run audits locally before pushing — fastest feedback loop, keeps CI noise low.
+ Install dependencies in the repository and commit the lockfile — avoid
+ requiring an unspecified latest global package.
- **`package.json` scripts:**
```json
{
"scripts": {
- "test:a11y": "lhci autorun && npx playwright test",
- "test:a11y:local": "lhci collect --url=http://localhost:3000 && lhci assert"
+ "test:a11y": "playwright test tests/accessibility",
+ "test:a11y:update": "playwright test tests/accessibility --update-snapshots",
+ "test:a11y:ci": "playwright test tests/accessibility --reporter=line,json"
}
}
```
- **Install once:**
```bash
- npm install -g @lhci/cli
- npm install -D @playwright/test @axe-core/playwright
+ npm install --save-dev @playwright/test @axe-core/playwright
+ npx playwright install --with-deps chromium
+ npm run test:a11y
```
- ---
-
- ## Moderate: Shift-Left Accessibility Strategy
-
- Prevent issues from entering commits in the first place. Catch problems in order
- from fastest to slowest feedback:
-
- 1. **In-editor / local lint** — framework-specific a11y lint rules run as you type.
- 2. **Pre-commit gate** — run checks on changed files only; block commit when checks
- fail. Use `pre-commit` (Python) or `husky` + `lint-staged` (Node). Keep total
- runtime ≤ 30–60 seconds.
- 3. **PR gate** — re-run in CI for trust and consistency; fail PR on blocking
- regressions; publish artifact links.
- 4. **Scheduled depth scans** — nightly/weekly deeper scans; auto-label findings;
- trend metrics over time in `ACCESSIBILITY.md`.
-
- **Suggested Definition of Done addition:**
- > No UI-impacting commit is accepted unless local/pre-commit accessibility checks
- > pass and PR CI accessibility checks are green.
-
- **Policy model:**
- * **Fast fail locally** — contributors get immediate feedback before push.
- * **Strict PR enforcement** — no merge with blocking accessibility failures.
- * **Transparent metrics** — show pass rate, open defects, and remediation trend in `ACCESSIBILITY.md`.
- * **Waiver discipline** — only time-bound waivers with explicit owner and expiry.
+ Use `npm ci` in CI so the resolved dependency graph matches the reviewed lockfile.
---
- ## Moderate: AI-Assisted Remediation Loop
-
- Close the detect → fix gap using GitHub's accessibility scanner and the Copilot
- coding agent:
-
- 1. The scheduled scan (workflow B above) uses `github/accessibility-scanner@v3`
- and creates issues labelled `accessibility`.
- 2. A companion remediation workflow watches for that label event and passes the
- issue to a Copilot coding agent.
- 3. The agent locates the offending code, applies a minimal fix, and opens a
- **draft pull request** linked to the original issue.
- 4. A human reviews and merges the draft PR.
-
- Copy [`examples/AGENT_REMEDIATION_WORKFLOW.yml`](https://github.com/mgifford/ACCESSIBILITY.md/blob/main/examples/AGENT_REMEDIATION_WORKFLOW.yml)
- to `.github/workflows/accessibility-remediation.yml` to enable this loop.
-
- Covered violation types: `image-alt`, `label`, `link-name`, `heading-order`,
- `color-contrast`, `aria-required-attr`.
+ ## Moderate: Shift-Left Strategy
- > Requires a GitHub Copilot subscription with the coding agent feature enabled
- > and **Settings → Copilot → "Allow Copilot to create and approve pull requests"**
- > turned on.
+ Catch problems in order from fastest to slowest feedback: (1) in-editor/local
+ lint with framework-specific a11y rules, (2) pre-commit gate on changed files
+ only (`husky`+`lint-staged` or `pre-commit`; keep total runtime ≤30–60s), (3)
+ PR gate re-running in CI, failing on blocking regressions, publishing artifact
+ links, (4) scheduled deeper scans with auto-labeled findings and trend
+ metrics. No UI-impacting commit should be accepted unless local/pre-commit
+ checks pass and PR CI checks are green.
---
- ## Tool Comparison
-
- | Approach | Finds WCAG rule violations | Finds announcement quality issues | Works for SVG / canvas | CI-friendly |
- |:---|:---:|:---:|:---:|:---:|
- | axe-core | ✅ | ❌ | Limited | ✅ |
- | Lighthouse | ✅ | ❌ | ❌ | ✅ |
- | Playwright aria snapshots | Partial | ✅ | ✅ | ✅ |
- | Guidepup virtual screen reader | ❌ | ✅ | ✅ | ✅ |
- | Manual screen reader testing | Partial | ✅ | ✅ | ❌ |
+ ## Baselines, Suppressions, and Exceptions
- No single tool catches everything — use the approaches together.
+ Prefer preventing new regressions while existing findings are assigned and
+ remediated. A baseline is a temporary inventory, not proof existing barriers
+ are acceptable. Every suppression/exception should include: the exact rule/
+ file/element/route/task affected; reason and supporting evidence; user impact
+ and available workaround; an accountable owner and linked issue; an expiry or
+ review date; and the narrowest practical scope. Do not suppress an entire
+ rule when one reviewed instance is the exception. Do not update snapshots or
+ baselines solely to make CI green.
---
- ## Governance and SLAs
+ ## Sustainable Automation
- * **Critical failures:** any page below 100 % Lighthouse Accessibility blocks the build.
- * **Performance budget:** any page below 100 % Lighthouse Performance blocks the build.
- * **Triage:** scheduled scan findings must be converted to GitHub Issues.
- If an issue remains open, subsequent scheduled scans are paused (alert-fatigue guard).
- * **SLA:** triage critical failures within one business day; serious within one sprint.
+ Run fast targeted checks on every PR; reserve full crawls and large browser
+ matrices for schedules, releases, or high-risk changes. Cache reviewed
+ dependencies where it improves speed without hiding updates. Cancel obsolete
+ runs when it can't discard required evidence. Retain artifacts only as long
+ as necessary. Avoid duplicate scanners providing the same evidence. Use
+ deterministic tooling before AI when it can answer the question.
---
## Definition of Done Checklist
- * [ ] Lighthouse CI workflow runs on every PR and push to `main`
- * [ ] axe-core (via Playwright) runs on every PR covering WCAG 2.x AA tags
- * [ ] Both light and dark colour schemes tested
- * [ ] Mobile emulation enabled in Playwright tests
- * [ ] `.lighthouserc` score threshold set to `warn ≥ 0.9` or `error ≥ 1.0`
- * [ ] Scheduled scan workflow present with alert-fatigue guard
- * [ ] Scan failures auto-labelled `accessibility` and converted to issues
- * [ ] Accessibility tree tests in place for SVG, custom widgets, and live regions
- * [ ] Local `test:a11y` script documented in contributing guide
- * [ ] `audit-report.json` artifact uploaded on manual deep-crawl run
- * [ ] Pre-commit / lint-staged accessibility check present and runs on changed files
- * [ ] Shift-left metrics (pass rate, open defects) tracked in `ACCESSIBILITY.md` table
+ * [ ] Affected user tasks, pages, components, and states are defined for each check
+ * [ ] Dependencies, browsers, actions, and configuration are versioned and reviewable
+ * [ ] Final rendered output is tested (not source templates alone)
+ * [ ] Applicable themes, preferences, viewports, and interaction states are covered
+ * [ ] Automated results are interpreted within their documented limitations
+ (score ≠ conformance; tool severity ≠ project severity)
+ * [ ] Required manual and AT tests are completed or precisely handed off
+ * [ ] New Critical and Serious barriers are not knowingly introduced
+ * [ ] Findings include reproducible evidence and an owner
+ * [ ] Suppressions and exceptions are narrow, justified, tracked, and dated
+ * [ ] Workflow permissions, untrusted input handling, secrets, and artifacts are reviewed
+ * [ ] Action references use reviewed full commit SHAs, not tags
+ * [ ] `persist-credentials: false` set where later steps don't need Git access
+ * [ ] `pull_request_target` avoided for workflows executing untrusted PR code
+ * [ ] AI remediation workflow is manual-trigger-only, read-only token scope,
+ denies shell/URL/memory tools, and never commits/pushes/merges itself
+ * [ ] The final workflow is validated in a test branch before production use
+ * [ ] Documentation and this skill are kept in sync (not automatic)
---
## Key WCAG Criteria (automation coverage)
* 1.1.1 Non-text Content (A) — caught by axe-core / Lighthouse
* 1.3.1 Info and Relationships (A) — caught by axe-core
- * 1.4.3 Contrast Minimum (AA) — caught by Lighthouse / axe-core
+ * 1.4.3 Contrast Minimum (AA) — caught by Lighthouse / axe-core (visual inspection still needed)
* 4.1.2 Name, Role, Value (A) — caught by axe-core + aria snapshots
* 4.1.3 Status Messages (AA) — partially caught by axe-core
- > Automation covers ~30–40 % of WCAG issues. Pair with manual and assistive
- > technology testing to achieve full conformance.
-
- ---
-
- ## Alternative Tools
-
- * **[AccessLint](https://github.com/accesslint):** GitHub App for inline PR comments — useful backup for code-review-time catches.
- * **[Open-Scans](https://github.com/mgifford/open-scans):** External scans using multiple engines against a live URL.
- * **[Lighthouse CI server](https://github.com/GoogleChrome/lighthouse-ci):** Historical tracking and dashboards.
- * **[CivicActions: Scaling Automation](https://accessibility.civicactions.com/posts/how-we-scale-inclusive-website-content-with-automated-testing-and-open-source-tools):** Enterprise-scale a11y philosophy.
+ > Automation covers roughly 30–40% of WCAG issues at best. Pair with manual
+ > and assistive technology testing — see `skills/manual-testing/SKILL.md`.
---
## References
* [Full best practices guide](https://github.com/mgifford/ACCESSIBILITY.md/blob/main/examples/CI_CD_ACCESSIBILITY_BEST_PRACTICES.md)
* [Shift-left automation guide](https://github.com/mgifford/ACCESSIBILITY.md/blob/main/examples/SHIFT_LEFT_ACCESSIBILITY_AUTOMATION.md)
* [GitHub Accessibility Scanner integration](https://github.com/mgifford/ACCESSIBILITY.md/blob/main/examples/GITHUB_ACCESSIBILITY_SCANNER_INTEGRATION.md)
+ * [GitHub Actions Secure Use Reference](https://docs.github.com/en/actions/security-for-github-actions/security-guides/security-hardening-for-github-actions)
+ * [GitHub: Script Injections](https://docs.github.com/en/actions/concepts/security/script-injections)
* [Lighthouse CI documentation](https://github.com/GoogleChrome/lighthouse-ci)
* [axe-core/playwright](https://github.com/dequelabs/axe-core-npm/tree/develop/packages/playwright)
* [Playwright aria snapshots](https://playwright.dev/docs/aria-snapshots)
* [Guidepup virtual screen reader](https://www.guidepup.dev)
- * [github/accessibility-scanner action](https://github.com/marketplace/actions/accessibility-scanner)
- * [WebDriver BiDi accessibility tree spec](https://github.com/w3c/webdriver-bidi/issues/443)
+ * [GitHub Accessibility Scanner](https://github.com/github/accessibility-scanner)
+
+ > **Standards horizon:** These rules target WCAG 2.2 AA.
+ > Monitor: <https://www.w3.org/TR/wcag-3.0/>