Ask most managers how accessible their product is and they will quote a single score from a scan run once, months ago. Ninety-two percent, maybe. Or a letter grade pulled from a browser plugin somebody ran the week before a board meeting.
That number was already incomplete the day it was generated. It told you where you stood on one Tuesday, against one snapshot of your code, checked by one tool that only sees part of the picture. It did not tell you whether things are getting better or worse. It did not tell you where to spend the next sprint. It is a photograph being used to answer a question about motion.
The metric that actually matters is much simpler, and it is a trend line, not a single point.
The Stat: Deque's own published research puts automated accessibility scanning at catching roughly 57.38% of WCAG issues on its own - a single automated pass/fail score is measuring a partial view of the real issue count by design. (Source: Deque)
Why the single score fails you
A pass/fail score compresses everything into one number: the color contrast issue that annoys sighted users, the missing form label that makes checkout impossible for a screen reader user, the decorative icon with a redundant alt attribute. All of it gets folded into one blended figure, as if a checkout blocker and a cosmetic label were the same size problem.
Worse, that number is built on tooling that was never designed to see the whole board. Automated scanners are excellent at catching a defined slice of issues, fast and cheaply, which is exactly why they exist. But Deque's published research on automated accessibility testing coverage is direct about the limits of that slice. Things like logical reading order, whether an error message is actually announced to a screen reader, or whether a custom widget behaves the way a keyboard user expects, mostly require a human to check. A single score reported to leadership is quietly reporting only the part a machine can see.
There is a structural reason for this too. WCAG itself was never built around one number. The W3C's own WCAG conformance requirements define three separate levels, A, AA, and AAA, each applied against a defined scope of pages and processes. Conformance is a statement about which success criteria are met within a specific scope, not a blended percentage. When a report reduces that model to "we are at 91%," it is translating WCAG into a shape WCAG was never designed to take.
That mismatch matters more than it sounds like it should, because it shapes what a manager thinks they are allowed to ask for. If you believe accessibility is a single percentage, the natural leadership move is to ask engineering to "get the number up," which invites whatever fix is fastest and cheapest rather than whatever fix actually removes a barrier. A severity-tagged, criterion-aware view invites a better question: which specific success criteria are failing, in which specific parts of the product, for which specific group of users. That is a question a team can actually act on.
It also changes how the conversation lands with engineering. Nobody on a product team gets excited about chasing an abstract percentage two points higher. Engineers get engaged when the ask is concrete: this checkout flow has four critical issues blocking screen reader users from completing a purchase, here they are, here is how we will know when they are closed. A trend line built from real severity data gives you that conversation for free. A single blended score never will.
What to report instead
Track open issues, broken out by severity, at a regular cadence. That is the whole idea. It is unglamorous, and that is exactly why it works.
Here is what that structure looks like in practice:
| Severity | What it typically blocks | Example |
|---|---|---|
| Critical | Completing a core task at all | Checkout button has no accessible name; form submits silently on error |
| Serious | Significant friction or exclusion | Low contrast text on primary CTA; missing focus indicator on nav |
| Moderate | Slower, more effortful use | Redundant alt text; heading levels skip from H2 to H4 |
| Minor | Polish, rarely blocking | Decorative image missing empty alt attribute |
Reported once a month (or once a sprint, if your team ships that fast), that table becomes a trend line instead of a snapshot. A manager looking at four months of this data can answer questions a single score never could:
- Is the total number of open critical issues going down, staying flat, or growing as new features ship?
- Are new features introducing new critical or serious issues faster than the team is closing old ones?
- Is remediation concentrated on cheap, cosmetic fixes while blocking issues sit untouched?
- Does the severity mix match where real users are actually getting stuck, not just what the scanner flagged first?
- Is the backlog of known issues actually shrinking, or just getting reshuffled between sprints?
None of those questions can be answered by "we scored 91% last quarter." All of them can be answered by a severity-tagged, dated issue log that gets reviewed on a schedule.
How to start tracking it
The mechanics are simple enough that the hard part is discipline, not tooling.
- Establish a real baseline. Not a scanner run in isolation, a mixed review that includes manual checks against the parts automated tools cannot reliably see: keyboard traversal, screen reader announcements, focus order, and error handling.
- Tag every open issue by severity, using a consistent rubric like the one above, so the same kind of problem gets the same label every time regardless of who logged it.
- Report the counts on a fixed cadence. Monthly is usually right for leadership; sprint-level is better for the team actually doing the fixing.
- Chart it. A simple line per severity tier, over time, is more persuasive in a leadership meeting than any single percentage will ever be, because it shows direction, not just position.
- Revisit the baseline periodically. New features and redesigns introduce new surface area to check, so the issue count should be re-baselined, not just decremented forever.
None of these steps require exotic tooling or a large new budget line. Most teams already have an issue tracker; the change is in how issues get tagged and how often the totals get pulled into a report someone with budget authority actually reads. The discipline is the hard part, not the mechanics. A severity tag that gets applied inconsistently, or a monthly report that quietly slips to quarterly the first time the team gets busy, will erode the trend line faster than any tooling gap will.
It also helps to separate the metric you report externally, to leadership or a board, from the working list your engineering team triages daily. Leadership generally needs the four-number rollup by severity, tracked monthly. The team doing the fixing needs the full backlog, with enough detail to reproduce and close each issue. Trying to serve both audiences with the same document is usually how the reporting cadence quietly dies.
We've written before about why teams that fixate on a perfect score often burn out chasing the wrong target. If that sounds familiar, it's worth reading why chasing a 100% score is the wrong goal in the first place before you build your next reporting deck around one.
The trend line is the whole point
A single score answers "how are we doing right now, according to one tool." A tracked, severity-tagged issue count answers "are we actually getting better, and where should the next sprint go." Only one of those is useful to a manager trying to run a program instead of just report a number.
If your team doesn't have a real baseline yet, that's the actual first step, not another automated scan you'll quote once and forget. You can get a real baseline audit to start tracking this metric from, the kind that combines automated coverage with the manual checks that catch what tools miss. Get a real baseline audit so you have a number worth tracking, month over month, instead of a score you dust off once a year. If you'd rather talk it through first, the team at experts@wcag.world can walk you through what a severity-tagged baseline would look like for your product.