Your accessibility score is 95 and your site is still unusable

A developer runs axe on the checkout page. Score: 95. Green across the board. The team ships with confidence.

Three days later, a blind user tries to complete a purchase. The screen reader announces "button" seventeen times with no context. The focus jumps from the payment form to the footer after selecting a shipping option. A keyboard user gets trapped inside a date picker and cannot reach the submit button.

The score was 95. The site was unusable.

This is the accessibility version of a problem QA engineers know well: high test coverage that tests nothing meaningful. You can have 80% code coverage where every test checks that a function returns without throwing, and zero tests that verify the function actually does what it should. Automated accessibility scanners have the same blind spot. They verify the presence of attributes without testing whether those attributes create a usable experience.

What automated scanners actually test

Automated WCAG scanners like axe, WAVE, and Lighthouse operate on a single DOM snapshot. They check:

• Whether images have `alt` attributes (not whether the alt text is useful)
• Whether form inputs have labels (not whether the label makes sense in context)
• Whether color contrast ratios meet minimums (not whether the text is readable against complex backgrounds)
• Whether ARIA roles exist (not whether they describe the actual behavior)
• Whether heading hierarchy follows numerical order (not whether it reflects the content structure)

These are real checks. They catch real problems. But they operate on a single page at a single moment in time, with no concept of user interaction.

The coverage gap is enormous

WCAG 2.2 Level AA contains 55 success criteria. Research from the Government Digital Service (UK) and accessibility practitioners like Karl Groves has consistently shown that automated tools can fully test roughly 30-40% of those criteria. The remaining 60-70% require human judgment, interaction testing, or multi-step workflow validation.

| What automated tools catch | What they miss | |---------------------------|----------------| | Missing alt text | Alt text that says "image" or "banner_final_v3.jpg" | | Missing form labels | Labels that don't match what the input actually collects | | Color contrast below 4.5:1 | Text over gradients, images, or video backgrounds | | Missing ARIA landmarks | ARIA roles that contradict the native HTML semantics | | Heading level skips | Heading text that doesn't describe the section content | | Missing language attribute | Content sections in a different language than declared | | Empty buttons | Buttons with icons whose accessible names are wrong |

The second column is where real users fail. And no automated scanner can detect these issues because they require understanding intent, context, and the complete interaction flow.

Five failures that score 95 and break everything

1. The keyboard trap

A modal dialog opens when a user clicks "Add to cart." Keyboard users press Tab expecting to move through the dialog. Instead, focus cycles between two elements inside the modal and the browser's address bar. There is no way to reach the close button. There is no way to continue shopping. The user is stuck.

Automated scanners do not detect this. The modal has proper `role="dialog"`, an `aria-label`, and visible focus indicators. Every DOM-level check passes. But the focus management JavaScript is broken, and no snapshot-based tool can know that because it never pressed Tab inside the modal.

WCAG 2.1.2 (No Keyboard Trap) is marked `automatable: false` for exactly this reason. You have to interact with the component to find the trap.

2. The focus order disaster

A multi-step form has five sections. Visually, they flow top to bottom. But the DOM order, dictated by a CSS grid layout with `order` properties, sends focus from section one to section four to section two. A sighted mouse user sees a logical form. A keyboard user or screen reader user encounters chaos.

WCAG 2.4.3 (Focus Order) requires that focus sequence preserves meaning and operability. Automated tools can parse the DOM order, but they cannot determine whether that order matches the visual presentation across every layout breakpoint and interactive state.

3. The screen reader confusion

A React single-page application updates the page content when navigating between routes. The URL changes. The visible content changes. But no announcement is made to assistive technology. A screen reader user clicks "View order history" and hears nothing. They don't know the page changed. They don't know where focus went. They start exploring from the top of the page, re-reading the entire header and navigation.

The automated scanner sees a well-structured page with proper headings, landmarks, and labels. Everything passes. But the transition between pages, the moment that matters most, is invisible to the tool.

4. The timeout without warning

A government benefits application has a 15-minute session timeout. After 14 minutes of filling out a long form, the session expires. The page redirects to the login screen. The form data is gone.

WCAG 2.2.1 (Timing Adjustable) requires that users be warned before a timeout and given the option to extend. An automated scanner testing the form page sees a well-labelled form with correct ARIA attributes. Score: high. But the scanner never waited 14 minutes to see what happens.

5. Dynamic content that announces nothing

A search-as-you-type input shows results below the field as the user types. Sighted users see results appear instantly. Screen reader users hear nothing because the results container lacks `aria-live="polite"` or uses it incorrectly (like wrapping the entire page region in a live area, causing the screen reader to re-announce everything on every keystroke).

The scanner checks: form input has a label (yes), results list has proper markup (yes), interactive elements are focusable (yes). Pass. But the connection between typing and receiving results, the core experience, is broken for screen reader users.

Why this mirrors the test coverage problem

Software QA teams learned this lesson with test coverage metrics years ago. A codebase with 80% line coverage can have zero meaningful assertions. Tests that call functions without checking return values. Integration tests where the database is mocked so thoroughly that the test can never fail. End-to-end tests that click through a flow but never verify the outcome.

The parallel to accessibility testing is exact:

| Testing anti-pattern | Accessibility equivalent | |---------------------|------------------------| | Tests with no assertions | Checking for `alt` attribute without validating content | | Mocked dependencies that always succeed | Testing a static DOM without user interaction | | High line coverage, no branch coverage | Checking individual pages, never testing journeys | | Tests that pass in CI, fail in production | Scans that pass on desktop, miss mobile interaction issues | | Snapshot tests that nobody reviews | VPAT documentation based on automated scan output alone |

In both cases, the metric creates a false sense of security. Teams look at the number, assume quality, and stop investigating.

What actually catches these problems

The issues described above share a common trait: they only appear during real interaction. You have to press keys, follow a workflow, wait for timeouts, and use assistive technology.

Journey-based testing

Instead of scanning individual pages, test complete user workflows. A checkout flow is not five isolated pages. It is a sequence where:

• Focus must transfer correctly between steps
• Form state must persist across navigation
• Error messages must be announced when they appear
• Timeout warnings must be presented accessibly
• The final confirmation must be reachable by keyboard alone

Auditi is built around this concept. You define user journeys with sequential steps, then test each WCAG criterion in the context of that journey. A criterion like "Focus Order" is not tested on a page: it is tested across the transition from shipping address to payment method. The step-by-step workflow tracks which criteria were tested per step, who tested them, and what evidence they collected.

This matters because a focus order bug that only appears when moving between form sections will never show up in a page-level scan.

Manual testing with structured guidance

For the 60-70% of WCAG criteria that cannot be automated, testers need clear instructions. Each criterion in Auditi includes specific testing steps. For WCAG 2.1.1 (Keyboard), the guidance is: "Navigate entire page using only keyboard. Check custom controls work with keyboard." For WCAG 2.4.3 (Focus Order): "Verify focus sequence follows reading order."

This turns accessibility testing from "does this attribute exist" into "does this interaction work." Every result captures pass, fail, needs review, or not applicable, with severity ratings and tester notes attached as evidence.

Multi-standard compliance in one audit

Different industries face different requirements. Healthcare software must meet FDA 21 CFR Part 11 for electronic records validation. Pharmaceutical companies in Europe need EU GMP Annex 11 compliance. Government agencies require Section 508. Some organizations are now navigating the EU AI Act's accessibility provisions.

Running separate audits against each standard is wasteful. Auditi supports WCAG 2.1, 2.2, 3.0, FDA 21 CFR Part 11, EU Annex 11, and EU AI Act in a single project, so teams can audit once and generate compliance documentation for each framework they need.

VPAT generation from real test data

A Voluntary Product Accessibility Template (VPAT) is only as good as the testing behind it. VPATs generated from automated scan results alone will show criteria as "Supports" when the automated check passed, even if the criterion requires manual verification.

Auditi generates VPATs from the combined results of manual and automated testing, organized by WCAG principle (Perceivable, Operable, Understandable, Robust), with tester remarks and evidence counts per criterion. A VPAT that shows "Does Not Support" with specific evidence is more valuable than one that shows "Supports" based on a scanner that cannot test the criterion.

The real cost of false confidence

When an automated tool reports 95 and the team stops there, the consequences are predictable:

Legal exposure. Courts have consistently ruled that automated scanning alone does not constitute a good-faith effort toward accessibility. The number of ADA-related web accessibility lawsuits exceeded 4,600 in 2023 (UsableNet). A green dashboard does not hold up in court if a plaintiff demonstrates that a keyboard user cannot complete a core task.

User abandonment. The Click-Away Pound Survey (UK) found that 69% of disabled users will leave a website that presents barriers, taking their business to a competitor. They do not file a complaint first. They just leave.

Remediation cost multiplier. Fixing accessibility issues discovered post-launch costs 10-30x more than addressing them during development. Fixing issues discovered during a lawsuit costs even more, plus legal fees.

Reputation. Accessibility failures documented publicly (through lawsuits, social media, or advocacy groups) create lasting reputation damage that no PR response can fully reverse.

How to stop trusting the number

If your team currently relies on automated accessibility scores, here is a practical path forward:

• Keep the automated scans. They catch real issues. Do not remove them from your CI/CD pipeline. Just stop treating them as the final word.

• Identify your critical journeys. What are the 5-10 user workflows that define your product? Signup, checkout, account management, search, content consumption. These are your accessibility testing priorities.

• Test those journeys manually. Navigate each one with only a keyboard. Use a screen reader (VoiceOver, NVDA, or JAWS) for at least the top three journeys. Time yourself: can a keyboard user complete the flow in a reasonable timeframe?

• Document what you find. Every manual test result should capture the criterion, the outcome, the evidence, and who tested it. This is the data that makes VPATs credible and demonstrates good-faith effort.

• Make it continuous. Accessibility is not a one-time audit. Every major feature release changes the interaction model. Journey-based testing should be part of your QA cycle, not an annual checkbox.

BetterQA has been doing independent quality assurance for over 50 companies across 24+ countries since 2018. The pattern is always the same: automated tools create confidence, manual testing reveals reality. Accessibility is no exception.

The score is a starting point, not a destination

A 95 accessibility score means your automated checks are in good shape. That is worth something. It means you have alt attributes, form labels, ARIA landmarks, and sufficient color contrast.

But it does not mean a keyboard user can navigate your checkout flow. It does not mean a screen reader user understands your page transitions. It does not mean a user with cognitive disabilities can recover from an error in your multi-step form.

The score tells you about the code. It does not tell you about the experience.

Testing the experience requires testing the journey. Auditi exists because that gap needs to be closed, not with more automated scans, but with structured human testing that follows real users through real workflows.

Your score is 95. Now test whether your site actually works.