Ask anyone building assistive tech what's coming next, and you'll hear a version of the same pitch. Point an AI model at a screen, and it will just describe what's there, out loud, the way a sighted friend would over the phone. Apps like Seeing AI and Be My AI already do a version of this today, and a lot of product roadmaps quietly assume the rest of that problem is basically solved.
But "describe the screen" is doing a lot of work in that sentence. A screen isn't just shapes and colors. It's a set of specific, checkable facts: does this button have a name, does this error message actually say anything, is this text readable against its background. So we asked a narrower, more useful question. If you handed four different AI models the exact same broken screen, with the exact same real bugs hidden inside it, would they even agree with each other about what's wrong?
The Stat: In our own real test, all 4 of 4 AI models we asked to narrate the same broken screen correctly caught 2 of 3 real, deliberately injected accessibility bugs, but only 3 of 4 caught the one bug that required actually measuring contrast (2.21:1, a real WCAG AA fail) rather than just spotting an obviously missing label. (Source: our own real, reproducible test, detailed in the article below)
What We Actually Built
We didn't scrape a real product, and we're not naming a vendor as having failed anything. We built one static test image ourselves: a fake signup screen with exactly three real, deliberately injected accessibility problems, chosen because each one fails in a different way.
The first bug was a search button that was only an icon, with no visible text label anywhere nearby. The second was an email field shown in an error state using nothing but a red border, with no visible message telling anyone what actually needed fixing. The third was a legal disclaimer line, set in light gray text on a white background, that we didn't just eyeball. We ran it through the real WCAG relative-luminance contrast formula ourselves and measured it at 2.21:1. WCAG AA requires 4.5:1 for normal body text, so this wasn't a borderline judgment call. It was a clear, measurable fail.
Then came the part that actually matters for this article. We took that single image, gave it to four different AI models with no hints about what we'd planted, and asked each one to narrate the screen out loud, the same way an AI visual-description tool built for blind users would, then flag anything unclear or unusable non-visually.
The Results, Bug by Bug
The icon only button
Every single model caught this one. All four independently described the search control as an icon with no accompanying text, and all four flagged it as a real barrier for anyone who couldn't see the icon and infer its purpose from shape alone. This is, honestly, the easiest bug in the set. A missing label is a binary, visible absence. There's no ambiguity for a model to reason through, which is probably why the agreement was total.
The color only error state
Same story. All four models noticed the red border around the email field and, more importantly, all four explicitly called out that there was no visible text explaining what was wrong with the input. That distinction matters. Plenty of automated tools and casual reviewers will see a red box and assume the "error" part is communicated. These four models didn't stop there. They correctly reasoned that color alone conveys nothing to someone who can't perceive it, and said so in their narration.
The bug that needed a ruler, not just eyes
Here's where things got interesting. Three of the four models explicitly flagged the disclaimer text as a readability problem in their narration, correctly identifying it as low contrast. The fourth model described the disclaimer text accurately, word for word, but never once flagged it as a contrast issue. It simply narrated the words as if they were fully readable.
That's the one bug in our set with a hard number behind it. Not "this looks kind of light," but 2.21:1, independently measured, against a 4.5:1 requirement. And it's the one bug where the four models split.
The Real Finding Isn't What You'd Expect
It would be easy to write this up as either "AI screen readers are unreliable" or "AI screen readers are basically there now." Neither is honest. Zero of the four models invented a problem that wasn't actually in the image. Every single finding across all four narrations mapped back to one of the three real bugs we planted. Nobody hallucinated a fourth issue to sound thorough. That's a genuinely good result, and it deserves to be said plainly rather than buried under a more dramatic headline.
But the same four models, looking at the exact same pixels, reached different conclusions about the exact same piece of text. Three called it out. One didn't. That's not a story about one model being "worse." It's a story about what kind of bug an AI narration is reliable for catching, and what kind it isn't.
The two bugs every model caught were both structural and obvious once you're looking for them: a control with no name, an error with no message. The one bug that split the room required something else entirely, an actual measurement, run against an actual formula, compared to an actual threshold. Spotting "this text looks a bit light" is a judgment call. Knowing it fails at 2.21:1 against a 4.5:1 requirement is a calculation. AI narration, at least in this small test, was consistent at the first kind of task and inconsistent at the second.
Why This Matters More Than a Leaderboard
If you're building or buying a product that leans on AI narration as an assistive layer, that inconsistency is the actionable part. A tool that describes your screen out loud can be a genuinely useful companion for a blind user navigating an unfamiliar interface, and nothing in this test suggests otherwise. But it is not, and shouldn't be marketed as, a substitute for an actual automated contrast check that runs the same formula against every pixel every single time, with zero variance between runs.
Real screen reader users aren't a hypothetical audience here either. WebAIM's own survey of screen reader users has spent a decade tracking exactly how people who rely on this technology daily actually navigate the web, and it's worth reading if you want to understand the audience behind these bugs rather than an abstraction of them. The W3C's perspective videos are another good, short way to see why a missing label or an unreadable disclaimer isn't a minor cosmetic detail to the person who hits it.
If you'd rather not wait to find out which of your own screens have a version of these three bugs sitting in them right now, running an actual automated accessibility check against your real pages will catch the measurable ones (like that 2.21:1 contrast fail) with total consistency, every time, which is precisely the thing our fourth model didn't do.
The Mirror Image of a Problem We Already Wrote About
We've also written about AI agents that click, fill, and submit forms on your behalf and quietly break things for real screen reader users along the way. That piece is about AI acting on a page. This one is about AI describing a page. They're opposite mechanisms, but they land on the same underlying truth: AI sitting between your interface and a blind user, in either direction, is not a guarantee that the interface itself is actually accessible. If anything, it raises the stakes on getting the underlying markup right, because you can't always predict which AI layer a given user is running on top of it.
We've also looked at what happens when you ask an AI model to build the form itself rather than just narrate it, in our piece on everything ChatGPT got wrong building an accessible form. Read together, the three articles paint a consistent picture: AI is a useful assistant at almost every stage of this problem, and a reliable substitute for careful, measured accessibility testing at none of them.
Where This Leaves You
Four models, one broken screen, three real bugs. Two of them got caught every time. One of them, the one with an actual number attached, only got caught three times out of four. That's not a condemnation of AI narration tools. It's a reminder that "an AI described it and didn't mention a problem" is not the same statement as "there is no problem."
If you want to know for certain whether your own pages have a contrast fail, a color-only error state, or an unlabeled control sitting somewhere in production, that's exactly what real automated and manual testing is built to catch with certainty rather than a three-out-of-four hit rate. Reach out to a real person on our team at experts@wcag.world, or start with a scan through AccessRadar and see what turns up on your own screens.