What automated accessibility testing catches, and what it can't

The line: what a machine can decide

An automated accessibility check can answer questions with a definite answer in the DOM. Does this image have an alt attribute. Do these two colours meet a contrast ratio. Does this input have a label associated with it. Is the document language declared. These are computable, they are not matters of opinion, and a tool that reports them is reporting facts.

It cannot answer questions that require knowing what the page means. Is this alt text an accurate description of the image, or is it the filename. Is this heading structure the actual outline of the content, or a set of sizes someone liked. Does this focus order match the visual order a sighted user follows. Those need a person, and no amount of rule tuning changes that.

That line is the whole of this guide. A checker runs 70 WCAG rules against every rendered page, which is a real and worthwhile pass - and it is the machine-decidable half. Treating a clean automated report as an accessible site is the single most common accessibility mistake an organisation makes, and it is usually made in good faith.

What the automated pass reliably catches

These are worth running on every page of every deploy, because they are unambiguous, they regress constantly, and each one blocks real users outright.

• Images with no alt attribute at all - a screen reader falls back to reading the filename
• Form inputs with no associated label, which makes a form unusable rather than merely awkward
• Text and background colours below the contrast ratio, which affects far more people than it is usually budgeted for
• Missing document language, which makes a screen reader pronounce the page with the wrong phonemes
• Heading levels that skip, which breaks the outline people navigate by
• Buttons and links with no accessible name - an icon-only control with no label is a control that cannot be described
• A viewport that disables zoom, which is a single attribute that breaks the page for anyone who needs to magnify it
• Frames with no title, and duplicate element ids that break label associations

The right place for these is a gate, not a report. They are cheap enough to run on every build, and a rule that has to be re-fixed every quarter was never really fixed.

What it misses, with examples

Every item below passes an automated check completely, and every one of them is a real barrier.

• Alt text that exists and is useless. alt="image" and alt="DSC_0421.jpg" both satisfy every automated rule ever written.
• Alt text on a decorative image. A purely ornamental graphic should have an empty alt so it is skipped; a described one makes a screen reader announce noise between every paragraph.
• Keyboard traps. A modal that takes focus and never gives it back is invisible to a static check and completely blocking to anyone not using a mouse.
• Focus order that does not match visual order, usually caused by CSS reordering a grid without the DOM following.
• Custom controls with correct ARIA and wrong behaviour - a div with role="button" that has no key handler passes the rule and does nothing when you press Enter.
• Content that only conveys meaning through colour: a red row and a green row with identical text.
• Video with no captions, or with automatic captions nobody has corrected.
• Error messages that appear visually but are never announced, so a screen reader user submits a form and is told nothing.
• Motion and animation with no reduced-motion alternative.
• A skip link that exists, satisfies the rule, and points at an element that is not there.

Notice the pattern: automated rules verify that a mechanism is present. Almost every item above is a mechanism that is present and wrong. That is why the fraction of issues a tool finds cannot be stated as one number - it depends entirely on whether a given site's failures are missing mechanisms or broken ones.

The manual pass that covers most of the rest

You do not need a full audit to close most of the gap. Four passes over the templates that matter - not every page, the templates - catch the majority of what automation cannot see, and none of them needs specialist tooling.

• Unplug the mouse. Reach every interactive element with Tab, activate each with Enter or Space, and make sure you can always see where focus is. Then open every modal and get back out again.
• Read the page with a screen reader. VoiceOver on macOS and iOS, NVDA on Windows - both free, both installed in under a minute. Listen to one full page rather than sampling; the problems are in the transitions.
• Zoom the browser to 200% and then to 400%. Reflow failures are common, obvious once seen, and completely invisible to automation.
• Read the alt text as a list, out of context. If a line does not describe what the image communicates, it is not alt text - it is an attribute that satisfies a rule.

Where a scan cannot decide, the honest thing for a tool to do is say so and leave the item for a person to sign off - dated, attributable, and part of the record - rather than quietly scoring the page as though the question had been answered.

Why automate at all, then

Because the machine-decidable half regresses. A manual audit is a snapshot of one afternoon; the next sprint adds an icon button with no label, a new image with no alt, and a card component whose contrast is a shade off. Automation is what stops the site sliding back between audits - it is a regression test, not an assessment.

And because it is not free to be wrong here. In the EU, the Accessibility Act applies to a wide range of consumer-facing digital services; in the US, ADA-based litigation over websites is routine. Neither is satisfied by a passing automated report, which is one more reason to know exactly what that report does and does not cover.

The right shape is both: automated rules on every build so nothing regresses silently, plus a periodic manual pass over the templates that matter, with the results recorded so the next person can see what was checked, when, and by whom.