wcag-compliance-reporter · git:20260725.b4ad92d · 2026-07-25 · sha256 b9e8ac292f1ab564
wcag-compliance-reporter git:20260725.b4ad92dA
Immutable. This exact content never changes and is served at /api/v1/blob/b9e8ac292f1ab564.
---
name: wcag-compliance-reporter
description: "Builds a per-page WCAG 2.2 compliance score report by aggregating output from one or more accessibility scanners (axe-core / pa11y / lighthouse / WAVE / IBM Equal Access), pivoting violations by Success Criterion (1.4.3 contrast, 2.4.7 focus visible, etc.), grouping by conformance level (A / AA / AAA), reporting per-page coverage gaps explicitly (the \"this page wasn't scanned\" failure mode), and emitting both an executive summary and a per-page drill-down. Use after a multi-page accessibility scan - pa11y-ci, axe across a sitemap, lighthouse-batch - when the team needs a shareable conformance report rather than a per-page tool dump."
---
# wcag-compliance-reporter
## Overview
WCAG 2.2 has 77 success criteria (SC) split across three
conformance levels: 25 at A, 24 at AA, 28 at AAA
([wcag-spec][wcag-tr]) - for a total of 49 SC at the AA level (which
is the typical legal / contractual target in the US, EU, UK).
[wcag-tr]: https://www.w3.org/TR/WCAG22/
Per [wcag-conformance][wcag-conf]:
[wcag-conf]: https://www.w3.org/WAI/WCAG22/Understanding/conformance
> "**Level AA**: 'satisfies all the Level A and Level AA success
> criteria'"
>
> "All success criteria at a claimed level must be satisfied with no
> exceptions."
That binary verdict ("conforms or doesn't") makes per-tool output
("axe found 17 violations in 5 pages") hard to act on - the team
needs to know **which SC at which level on which page** is failing
the conformance claim.
This skill builds that report.
## When to use
- A multi-page scan has produced output from one or more of:
axe-core (via `@axe-core/cli` or `axe-puppeteer`), pa11y (via
`pa11y-ci`), Lighthouse (via `lighthouse-batch`), WAVE,
IBM Equal Access.
- A stakeholder asks "are we WCAG 2.2 AA conformant?" and wants a
document, not a CLI dump.
- The team is preparing an accessibility audit / VPAT / ACR.
- A PR introduces UI changes and the team needs a "before vs after"
on conformance.
If only one tool is in use, the tool's native HTML report may
suffice; this skill is for **aggregation** across multiple tools or
multiple pages.
## How to use
1. **Gather scan output** from one or more scanners (axe-core, pa11y,
Lighthouse, WAVE, IBM Equal Access) across every page in scope.
2. **Normalize per tool** into one SC-keyed `Violation` shape, mapping each
tool's rule ID to the WCAG Success Criterion it covers (Step 1).
3. **Aggregate** the normalized violations by SC and by page (Step 2).
4. **Compute the conformance verdict per level** (A / AA / AAA): `conformant`,
`non-conformant`, or `unknown` - where `unknown` means the scan never
covered the SCs a claim at that level needs (Step 3).
5. **Check page coverage** against the `pages-to-scan.yaml` process spec and
flag any page that wasn't scanned (Step 4).
6. **Render the report** - the executive verdict plus per-SC and per-page
markdown (Step 5), then emit the machine-readable `compliance.json` and the
CI job that produces it (Machine output and CI, below).
## Step 1 - Author normalizers per upstream tool
Each scanner has its own JSON shape. The first step is normalizing
to a single intermediate format keyed by SC:
```typescript
interface Violation {
page: string; // URL or path
successCriterion: string; // e.g. "1.4.3"
level: 'A' | 'AA' | 'AAA';
ruleId: string; // tool-specific rule (e.g. axe "color-contrast")
selector: string; // CSS selector of failing element
message: string;
helpUrl?: string; // tool's docs
scanner: 'axe' | 'pa11y' | 'lighthouse' | 'wave' | 'equal-access';
}
```
Normalizer per tool maps tool-specific rule IDs to the SC they cover, and a
central `sc-mapping.json` holds rule-to-SC for every tool the report consumes.
The per-tool tag conventions (axe `tags`, pa11y codes, Lighthouse's internal
map), a worked `normalize_axe` example, and the `sc-mapping.json` shape live in
[references/tool-normalizers.md](references/tool-normalizers.md).
## Step 2 - Aggregate
```python
def aggregate(violations):
by_sc = defaultdict(lambda: defaultdict(list)) # sc -> level -> [violation]
by_page = defaultdict(list) # page -> [violation]
pages = set()
for v in violations:
by_sc[v['successCriterion']][v['level']].append(v)
by_page[v['page']].append(v)
pages.add(v['page'])
return {
'by_sc': dict(by_sc),
'by_page': dict(by_page),
'pages': sorted(pages),
}
```
## Step 3 - Conformance verdict per level
Per [wcag-conformance][wcag-conf], conformance is binary per level
("All success criteria at a claimed level must be satisfied with no
exceptions"). The verdict logic:
```python
def conformance(agg, level):
"""Returns 'conformant' | 'non-conformant' | 'unknown'.
Unknown if the scan didn't cover the SCs needed to claim conformance."""
failing_scs = [sc for sc, by_level in agg['by_sc'].items()
if any(v['level'] in level_set(level) for v in
[vv for lvl, vs in by_level.items() for vv in vs])]
if failing_scs:
return 'non-conformant', failing_scs
# Did the scan actually cover all SCs needed for this level?
expected_scs = SC_BY_LEVEL[level] # ground-truth list per WCAG 2.2
covered_scs = set(agg['by_sc'].keys()) # only SCs at least one tool checks
missing = expected_scs - covered_scs
if missing:
return 'unknown', sorted(missing)
return 'conformant', []
```
`level_set('AA')` returns `{'A', 'AA'}` because AA "satisfies all the
Level A and Level AA success criteria" ([wcag-conformance][wcag-conf]).
The "unknown" verdict is critical - automated tools cover roughly
30% of WCAG SCs. A green automated scan ≠ AA conformance. The
report must surface what wasn't scanned.
## Step 4 - Per-page coverage gaps
Equally important: did every relevant page get scanned at all? A
checkout flow with a missing scan on the payment page can't claim
conformance per [wcag-conformance][wcag-conf]:
> "**Complete Processes**: When a page is part of a multi-step
> process, 'all web pages in the process conform at the specified
> level or better.'"
Author maintains a `pages-to-scan.yaml`:
```yaml
processes:
checkout:
pages:
- /
- /products
- /cart
- /checkout
- /checkout/payment
- /checkout/confirm
account:
pages:
- /account/login
- /account/profile
- /account/orders
```
Compare scanned pages against the spec; flag missing ones in the
report.
## Step 5 - Render the report
Render the executive verdict first: a three-row table mapping each conformance
level to its verdict and the reason.
| Conformance level | Verdict | Reason |
|-------------------|---------|--------|
| A | ❌ non-conformant | 3 SC failing on 7 pages |
| AA | ❌ non-conformant | + 5 additional SC failing on 12 pages |
| AAA | ⚠ unknown | 14 of 28 AAA SC weren't checked by any scanner |
Below the verdict, the full report adds process-coverage, failures-by-SC,
per-page-detail, and coverage-gap sections - the coverage-gap table is where the
"automated tools cover ~30% of SCs, the rest is manual review" reality from
Step 3 lands. The complete multi-table sample report, every section with example
rows, lives in
[references/sample-report-format.md](references/sample-report-format.md).
## Machine output and CI
Once the markdown report renders, two operational blocks finish the pipeline: a
machine-readable `compliance.json` for downstream dashboards / ACR gates, and
the CI job that runs the scanners and aggregator on every push. Both templates
live in
[references/machine-output-and-ci.md](references/machine-output-and-ci.md).
## Anti-patterns
| Anti-pattern | Why it fails | Fix |
|---------------------------------------------------------------------------------|------------------------------------------------------------------------------------|-----|
| Reporting "0 axe violations" as "WCAG conformant" | Automated tools cover ~30% of SCs; the rest needs manual review. | Always include the "manual review required" section (Step 5). |
| Treating all violation severities equally | A contrast issue on a button isn't equivalent to a missing alt on a decoration. | Group by SC level (A vs AA vs AAA); within a level, count instances.|
| Aggregating across pages without page coverage spec | Missing pages don't show up; the team thinks they're covered. | Author `pages-to-scan.yaml`; report missing pages explicitly (Step 4). |
| Declaring conformance from a single-tool scan | Each tool has different SC coverage; a clean axe run can hide pa11y-detectable issues. | Use 2+ tools; deduplicate by `(page, SC, selector)`. |
| Per-tool report dumps with no SC mapping | Reviewer can't tell if tool-rule "color-contrast" maps to SC 1.4.3. | Maintain `sc-mapping.json` (Step 1). |
| Reporting AAA verdict as a fail without context | AAA is rarely the target; failing AAA isn't a failure for an AA-targeting team. | Make the target level explicit; report the other levels as "informational". |
| Ignoring "unknown" verdict (treating uncovered SCs as covered) | False conformance claim; potential legal exposure. | Surface the unknown verdict in bold (Step 3); list the unscanned SCs. |
## Limitations
- **Automated scans don't replace manual review.** Roughly 30% of
WCAG SCs are tool-detectable; the rest require human assessment
(cognitive load, alternative text quality, color-only meaning,
etc.).
- **No screen-reader reality check.** Tools find the absence of
ARIA; they don't verify that the screen-reader narration is
helpful. Pair with `screen-reader-test-author` from
`qa-accessibility`.
- **Per-tool drift.** Rule catalogs evolve; new tool versions may
add or remove SC coverage. Pin tool versions in CI; bump the SC
mapping in lock-step.
- **VPAT / ACR is a different document.** This report informs the
VPAT but isn't the VPAT itself - those documents have specific
formats (Section 508, EN 301 549) that go beyond WCAG.
## References
- [wcag-tr][wcag-tr] - WCAG 2.2 specification: 77 SC across A / AA
/ AAA, four POUR principles, 13 guidelines.
- [wcag-conformance][wcag-conf] - WCAG 2.2 conformance requirements:
binary per-level verdict, complete-processes rule, no-exceptions
rule.
- `junit-xml-analysis` (in the qa-test-reporting plugin) - reporter
for test execution (different domain, same PR-time
reporting shape).
- The `qa-accessibility` plugin's per-tool wrappers
(`axe-a11y`, `pa11y-a11y`, `lighthouse-a11y`, `wave-a11y`,
`ibm-equal-access-a11y`) - produce the upstream input this skill
consumes.
- Per-tool normalizers + the `sc-mapping.json` shape (Step 1 deep dive):
[references/tool-normalizers.md](references/tool-normalizers.md).
- The full multi-table sample report format (Step 5 deep dive):
[references/sample-report-format.md](references/sample-report-format.md).
- Machine-readable `compliance.json` output + the CI aggregation job:
[references/machine-output-and-ci.md](references/machine-output-and-ci.md).