We took fifteen real hero images from our own live blog, sampled systematically across the archive, and ran a real test on ourselves: generate alt text for each image completely blind, no slug, no article title, no context beyond the pixels themselves, then compare that generated text against the real, already-published alt text sitting in each article's frontmatter. We expected the test to be mostly about whether AI-generated alt text is accurate. It turned out to be about something more interesting: whether the alt text a team writes before an image exists actually survives contact with the image that eventually gets generated.
The Stat: 7 of 15 blind reads matched our published alt text closely. The other 8 diverged, and every single divergence traced back to the same root cause: the published alt text described the original image-generation prompt's intended concept, not what the image model actually rendered. (Source: our own real, reproducible 15-image test, described in full below)
The Method
Our published articles include a real heroImageAlt field in every article's frontmatter, written as part of the same process that generates the hero image itself, one descriptive sentence meant to stand in for the illustration if it fails to load or is read by a screen reader. For this test, we pulled fifteen of those hero images at random intervals across the live archive, converted them to plain PNGs, and looked at each one directly, with no slug, no title, and no prior alt text visible, writing a single blind description of exactly what was on the canvas. Only after writing all fifteen blind descriptions did we compare them against the real, already-published alt text for each image.
What Matched
Seven of the fifteen blind reads lined up closely with the published alt text: a certification ribbon badge with a crack through it above a browser window, an envelope with a clock and checklist beside it, two overlapping color swatch circles with an eyedropper, a graduation cap above a laptop screen, a chat bubble icon partially over a dashed focus outline, a columned government building beside a document, and a checkmark badge document with a faded second document behind it. In each of these cases, the blind description and the published alt text agreed on the core subject, the composition, and the key visual detail, describing the same image from two independent angles.
What Diverged, and Why
The other eight told a more interesting story. In every one of the eight, the mismatch was not a case of the blind read missing something the image actually contained. It was the published alt text describing the concept from the original image-generation prompt rather than what the image model had actually rendered onto the canvas.
The clearest example: an article about "eleven prompts that turn ChatGPT into an accessibility assistant" had published alt text describing "a chat prompt bubble branching into eleven small labeled task icons." The actual rendered image showed a single checklist with four items inside one speech bubble, no branching, no eleven icons. Another article, about a fast-growing accessibility specialist career, had alt text describing "a job-listing card next to a rising line graph." The actual image showed a person looking up at a descending bar chart. A piece on WCAG burnout had alt text describing "a stressed checklist with an unreachable 100 percent target replaced by a downward trend line." The actual rendered image was a dial gauge beside a potted plant, no checklist, no percentage, no trend line anywhere in it.
The single most striking mismatch involved an article titled "why your perfect Lighthouse score still fails real users." Its published alt text described "a green checkmark score beside a tangled, broken pathway." The image model, generating the hero art, appears to have interpreted "Lighthouse" literally, as in the tool's name, and rendered an actual lighthouse tower beaming light at a laptop and a standing figure. A completely different, entirely coherent image, just not the one the alt text was written to describe.
The Partial Mismatches Worth Naming Too
Two of the eight divergences were partial rather than total. One article's published alt text described "an image icon with two speech bubbles, one labeled with a magnifying glass and one with a sound wave." The actual rendered image showed a shield-like badge shape rather than a literal image icon, but the two speech bubbles, the magnifying glass, and the sound wave detail were all genuinely present and correctly described. Another article's alt text described "a launch countdown screen with a checklist of six accessibility items beside a rocket icon." The rendered image did include a rocket and a checklist, correctly, but showed a shopping bag and a browser bar rather than a countdown screen, and the checklist had roughly four visible rows rather than a verifiable six. In both cases, the core objects were present and correctly named; the surrounding scene and specific counts were where the drift showed up, a smaller version of the same underlying pattern as the other six full mismatches.
What This Actually Tells Us
This was not a test of whether AI can write accurate alt text, in the sense of a model failing to correctly describe an image in front of it. Every blind read we produced accurately described what was actually on the canvas, including for all eight of the "mismatched" cases. The gap lived entirely upstream, in the alt text that had been written for the concept brief handed to an image generator, before confirming what that generator actually produced. This is a completely different, and in some ways more useful, finding than "AI alt text is unreliable." It suggests the opposite: alt text generated by looking directly at the finished pixels was accurate every single time it was tried in this test. The risk sits specifically in any alt text written before, or without, checking it against the actual delivered image, a risk that applies exactly as much to a human writer working from a brief as it does to an automated pipeline.
The practical lesson for any team, using AI-generated images or not: alt text needs to be validated against the shipped asset, not against the request that produced it. A well-written, specific-sounding description is not evidence it describes the right image, only a description checked against the actual file is.
We ran a related test on a different kind of visual content, whether AI vision models can accurately read our site's own SVG charts, in our chart-reading test, and our broader guide to using AI for alt text correctly covers the practical workflow this test points toward. Our existing piece on why bad alt text is killing your SEO and accessibility at once covers the cost of getting this wrong on both fronts.
If your own site has alt text that was written before final assets existed, or copied from an early draft that later changed, our free accessibility tools can help flag likely gaps, and our team is reachable directly at experts@wcag.world for a manual review. The W3C's decision tree for writing alt text and its broader alt text guidance are both worth reading in full.