AI · ARIA · Testing

Can AI Write Accessible ARIA? We Tested 4 Models on a Custom Combobox

An oxblood and cream editorial illustration of a search input dropdown widget with an aria-tag icon beside it, and a second identical-looking widget with the tag icon missing.
  • AI
  • ARIA
  • Testing

We gave four AI models one plain-language prompt: build a self-contained HTML, CSS, and JavaScript country-search autocomplete widget, eight sample countries, filter as you type. Nothing about accessibility, ARIA, screen readers, or keyboard support was mentioned anywhere in the prompt. We wanted an honest answer to a specific question: when accessibility is not asked for at all, does a modern AI model build it in anyway, as a default, or does it only show up when specifically requested?

The Stat: 3 of 4 models spontaneously built a fully WAI-ARIA-APG-compliant combobox with zero accessibility instruction in the prompt; 1 of 4, again the smallest and fastest model, produced a widget that works perfectly by mouse and eye and is completely invisible to assistive technology. (Source: our own real, reproducible 4-model test)

One combobox with ARIA roles present, one without Two nearly identical search input boxes with dropdown option lists below them. The left widget has a small tag icon labeled ARIA beside it. The right widget, visually identical, has no tag icon, representing a widget with no ARIA roles at all. ARIA

The Combobox Pattern, and Why It Is a Genuinely Hard One to Get Right

An autocomplete or "combobox" widget, a text input that filters and shows a list of matching options as the user types, is one of the more complex interactive patterns in the WAI-ARIA Authoring Practices Guide, specifically because it involves coordinating several pieces of state at once: whether the option list is currently open, which option is currently highlighted as the user moves through it with arrow keys, and how to communicate both of those things to a screen reader user who cannot see the visual highlight state at all. Getting this right by hand, even for an experienced developer, typically requires deliberately consulting the APG pattern rather than intuiting the correct ARIA structure from scratch.

Three Models Built the Full Pattern, Unprompted

Sonnet, opus, and fable each independently produced a combobox implementing the real WAI-ARIA APG combobox pattern: role="combobox" on the input, aria-expanded toggling as the list opens and closes, aria-autocomplete="list", aria-controls pointing at the option list, role="listbox" on the popup itself, role="option" with aria-selected on each item, and aria-activedescendant management so a screen reader announces the currently highlighted option as arrow keys move through the list. Escape and arrow-key support was implemented consistently across all three, along with a role="status" live region announcing result counts as the user typed.

Opus went further than the other two, additionally adding Home and End key support, prefers-reduced-motion handling for its dropdown animation, a real associated <label> element, and building its live-region announcements using safe DOM methods rather than innerHTML, avoiding a class of injection risk the other implementations did not specifically guard against. Fable's implementation, per its own generation summary, covered the same full keyboard set: arrows, Home, End, Enter, Escape, and Tab, with live-region announcements included.

None of this was requested. All of it was the model's own default behavior when asked to build a filter-as-you-type widget with no accessibility instruction anywhere in the prompt.

The Fourth Model Built Something Invisible

Haiku produced a widget that, by every visual and functional measure a sighted mouse user would apply, worked well: it filtered correctly as the user typed, supported mouse hover and click highlighting, and included a working clear button. It had zero ARIA attributes and zero semantic role anywhere in the markup. No role="combobox", no aria-expanded, no role="listbox", no live region. A screen reader landing on this widget announces a plain text input and, separately, an unlabeled list of plain text, with no indication that the two are connected, no indication that the list opens and closes in response to typing, and no way to know which option, if any, is currently highlighted. The widget is not degraded for a screen reader user. It is entirely absent as an interactive pattern; only its individual, disconnected pieces exist.

How the Three Complete Implementations Actually Differed From Each Other

It is worth noting that "three models built the full pattern" does not mean all three produced identical code. Sonnet's implementation used a straightforward aria-activedescendant approach with the option list always present in the DOM and toggled via aria-hidden. Opus additionally guarded against a subtle real-world bug the other two did not address at all: normalizing text input using Unicode NFD normalization before filtering, so a search for an accented country name still matches consistently regardless of how the accent character was encoded by the user's input method, a genuinely thoughtful detail nobody asked for. Fable's implementation, per its own build summary, was functionally equivalent to sonnet's in structure, with slightly different internal state management. None of the three complete implementations were byte-for-byte identical, but all three converged independently on the same correct semantic foundation, which is the more meaningful similarity than any specific implementation detail.

What This Suggests About Training Data, Cautiously

It is worth speculating carefully, and only carefully, about why three of four models defaulted to a correct, fairly intricate ARIA pattern without being asked. The WAI-ARIA Authoring Practices Guide is a widely referenced, publicly documented resource, and a combobox is one of its more thoroughly worked examples, appearing in countless tutorials, component libraries, and open-source implementations a model's training data plausibly included at scale. This is a reasonable hypothesis for why a well-known, well-documented pattern fared better than a bespoke one might, but it remains a hypothesis, not a confirmed mechanism, and it does not change the practical takeaway either way: verify the actual output, for any pattern, rather than assuming a model's general capability predicts its behavior on a specific, unverified case.

The Actual Finding, Stated Plainly

Three of four models, unprompted, reached for the correct, fairly intricate ARIA pattern by default. That is a genuinely encouraging result, and worth taking seriously: it suggests that for at least some current models, accessible markup is closer to a default behavior than an opt-in one, at least for a well-known pattern like a combobox. The fourth result is the one worth sitting with longer, because it is not a subtle gap. A widget that works flawlessly by every test a sighted developer would naturally run, click it, type in it, watch it filter, and is simultaneously completely unusable by a screen reader, is exactly the kind of gap that ships to production undetected, because the natural way anyone reviews a UI component (by looking at it and clicking it) will never surface this particular failure.

The practical takeaway is not "avoid faster or smaller AI models" as a blanket rule. It is that any AI-generated interactive component, regardless of which model produced it, is worth a real accessibility check specifically, the same unplug-the-mouse test we cover in the one screen reader test every developer skips, before shipping it, precisely because a component that looks and behaves perfectly by every visual and mouse-driven test can still be entirely invisible underneath.

For how the same four models handled a very different kind of accessibility task, fixing a broken table rather than building a component from scratch, see our companion piece, we asked 4 AI models to fix a broken data table. And for the general pattern of building keyboard support into any custom component, our keyboard navigation guide covers the broader discipline this test draws on, and our guide to when ARIA helps and when it hurts is worth reading alongside these results, since all four models here reached for ARIA specifically where a native element could not have provided the same semantics on its own.

If your team is using AI to help build custom UI components, our free accessibility tools are a fast first check, and our team is reachable directly at experts@wcag.world for a deeper manual review of anything AI-generated before it ships. The W3C's Combobox pattern and the WAI-ARIA specification itself are the primary sources behind the correct pattern described here.