Skip to main content

Search...

Accessibility in Apps: Why Do We Perform Manual Testing?

Mobile accessibility testing is mostly manual: VoiceOver, TalkBack and switch control can block each other, so the test matrix has to log every setting.

• • Updated: • 13 min read
Cover of the expert talk on 'Accessibility in Apps: Why Do We Perform Manual Testing?' with Thorsten Schröder, Dirk Haas and Richard Seidl.

Accessibility in mobile apps means that assistive technologies such as VoiceOver or TalkBack can reach every control in the app. What makes mobile accessibility testing hard is that settings for different groups, such as blind people, people with low vision and people with motor impairments, can get in each other’s way. There is no standard test matrix for this yet.

Key Takeaways

  • On mobile devices, assistive features for different disabilities exclude each other: TalkBack helps blind users but at the same time rules out sensible keyboard use for people with motor impairments.
  • A test matrix for mobile accessibility has to account for hundreds of switch combinations, so teams need to record exactly which settings were on and which were deliberately left off.
  • Accessible IT is not only a technical problem: defects start with designers and document authors, long before development.
  • The curb-cut effect makes accessibility features useful for everyone: high-contrast mode and AssistiveTouch also help people without disabilities, for example in bright sunlight or when they are distracted.
  • Making PDF documents accessible is technically solvable, but in practice organizations struggle with the sheer number of existing documents.

Mobile Accessibility Testing Is a Different Game from the Desktop

Anyone who tests software for accessibility knows the desktop setup: keyboard, mouse, maybe a camera. Mobile accessibility testing changes the rules. Once the application under test runs on an iPad or an Android device, the assistive technologies are already built into the operating system.

Thorsten Schröder and Dirk Haas test applications for the German statutory pension insurance, among them a mobile patient record on the iPad. Simply turning the device from portrait to landscape changed how the app behaved. Some elements no longer received focus when swiping with VoiceOver, which made them unreachable for blind users.

On a mobile device, you turn on features like VoiceOver on iOS or TalkBack on Android to have content read aloud. The app’s normal interaction, swiping and tapping, is then suddenly supposed to work through a keyboard. That is exactly where apps start behaving in unexpected ways.

Why VoiceOver and Switch Control Get in Each Other’s Way

The biggest problem in testing mobile accessibility is that settings for different disabilities rule each other out. What helps one group locks out another.

Here is a concrete case. For people with motor impairments there is Switch Control. On an iPhone, the camera can respond to head movements and move from one element to the next, with a sound, for example, triggering the selection. But Switch Control can’t run at the same time as VoiceOver. A blind person wouldn’t stand a chance with that combination.

For the test report, this is a real dilemma. All success criteria carry equal weight. None of them says whether a requirement applies only to blind users or only to users with motor impairments. The wording is the same for everyone. The tester has to document the restriction.

Who Makes the App Accessible, the Device or the Software?

Mobile devices keep adding alternative ways to operate them, and that raises a hard question for the assessment. One example shows the problem clearly.

WCAG has a success criterion for motion actuation: if you can discard data by shaking the device, you must also be able to do the same with a finger. Strictly speaking, the software should offer that alternative itself. On an iOS device, though, you can turn on AssistiveTouch, and the operating system then simulates shaking with a tap.

So is the software accessible or not? It doesn’t provide the alternative on its own, yet in that specific setup the task works. The person holding the device doesn’t care where the help comes from, as long as it works.

“Every release brings new accessibility features, and every time we have to take a fresh look, because there really is a lot of new stuff.”

(Dirk Haas)

Building a Test Matrix for Mobile Accessibility Testing

The practical route is systematic trial and error plus complete documentation of each configuration. There is no ready-made answer for every case.

Schröder and Haas sit down again and again, try out button and switch combinations and check what happens. If a certain switch is off and a blind person can operate everything under that condition, the criterion counts as passed for that type of disability. But that condition has to be written down explicitly. The next user might not have Switch Control turned off and could get stuck.

That makes the scope of each test report very narrow. There are now hundreds of possible switch combinations, far too many to cover them all. So the team documents both: the settings that were active and, explicitly, the ones that cause failures.

One fixed rule keeps the effort in check: within a single use case, nobody switches input methods halfway through.

“Either it works with one setting or it works with the other. But having to switch in the middle is not acceptable.”

(Thorsten Schröder)

The client supplies the relevant use cases. A tester can’t judge whether a blood pressure reading entered five times a day matters more than a number recorded once at admission.

Why Accessibility Is So Hard to Test Automatically

Accessibility tests of assistive technologies are entirely manual, and there is a logical reason. The test checks whether the assistive technology can reach the content at all.

A screen reader needs access to an element, its properties and its state before it can read it out. Test software would need exactly the same access. If a field can’t be reached, you can’t write an automated test for it either. And that missing access is often the very defect the test is meant to find.

Many popular test tools check things like valid HTML or HTML parsing. For a person with a disability who works with the application, that often doesn’t matter. Modern browsers handle invalid HTML reliably.

That shift shows up in a regulatory first. WCAG 2.2 is the first version to remove a success criterion: the one on parsing HTML. Tools built around exactly that check would lose a large share of their findings.

In practice, the change hasn’t taken effect yet. Laws and standards in force refer to WCAG as a whole, not to individual updates. The legal situation only changes once the European standard EN 301549 catches up and drops that point, and at European level such updates take a long time.

iOS and Android Are Converging, but Habits Stick

Apple’s early lead in assistive technology has largely disappeared. By now, both systems offer almost the same features.

Apple built in screen readers and assistive features from the beginning. Every new iOS version adds more, such as door or object detection in the Magnifier app, which uses the camera to identify people or things. For testers, that means more work, not less, because every new option makes the test matrix bigger.

One practical factor remains. Once people are used to a device and how it works, they rarely switch. That is true for everyone and even more so for people who depend on assistive technology.

What Is Built for People with Disabilities Helps Everyone

Features designed for people with disabilities are useful to everyone else, too. The curb-cut effect describes exactly that.

High contrast or a black-and-white mode doesn’t only help people with low vision. Stand outside in the sun with a phone screen full of glare, and high contrast is often the only way to see anything at all. Pink on green, and you see nothing.

People use these aids every day without knowing what they were originally built for. That is one of the strongest arguments for accessibility: it isn’t only for people with disabilities, it benefits a broad group of users.

Why Testers with Disabilities Are Involved, but Not in Every Test

People with disabilities are brought in selectively. A complete test with every disability group wouldn’t be affordable, because the effort would explode.

A blind person can’t see what they can’t see. If an element can’t be reached at all, they have no way of telling whether it was implemented correctly. A comprehensive test would need people who are hard of hearing, people with motor impairments, people with severe low vision and blind people, each with different degrees of impairment.

Schröder and Haas cover a broad range with their own testing and bring in specific expertise when they are unsure. A blind colleague, formerly a developer of screen reader software, has been involved in building a new design system from the start. His technical input is one benefit.

The other benefit is communication. If you work with a blind colleague, you can’t just send a screenshot without a description. That forces the team to communicate accessibly itself. The team also has its own presentation materials checked, because anyone who speaks about the topic should make their content at least as accessible.

It also shows how differently people work. Some blind users don’t read Braille and rely entirely on speech output, others work only with a Braille display. Neither really understands how the other works. Covering that range completely in testing is beyond any realistic scope.

The testers themselves deliberately take a naive approach and use the software as it comes. Experienced screen reader users have developed their own techniques, know where the problems are and work around them without thinking. That can’t be the benchmark, though, because then everyone else would need the same skills.

Accessibility Is Not Just a Technical Issue

The biggest misconception is that accessible IT is purely a matter of technology. If the structure of an application is wrong, even the best technical implementation won’t help.

If a design is poorly built and the technical structure doesn’t match what appears on screen, a blind person will never find their way around. That happens, for example, when someone uses layout tables. So the responsibility doesn’t sit with the developer at the end of the chain. It sits with the designer, the copywriters and the people who create the documents.

You can make a document look accessible in seconds by marking all its content as an artifact. The checking tool then reports green, because the screen reader has nothing left to read. For the blind user, the document is simply empty. Accessibility has to be considered from the top, in how programs are designed and in the time and resources that are made available.

Accessible Documents Are the Next Big Challenge

The topic everyone involved is wrestling with right now is providing accessible documents and forms. Moving from paper to accessible electronic content is a huge step.

The effort per document isn’t endless, but the number of documents almost is. Look inside your own organization and you will find a daunting pile of work instructions, forms and texts. Every one of them would need to be made accessible once. Partial automation with suitable tools is in progress, but that doesn’t change one thing: bad content stays bad, even when it is accessible.

The pressure isn’t spread evenly. Public bodies such as the pension insurance are already legally required to comply. Now it is becoming relevant for the private sector as well. Anyone who wants to run an online store in Europe without risking penalties from market surveillance authorities has to start implementing accessibility, or it will get expensive.

Frequently Asked Questions

Why can’t accessibility tests be easily transferred from the desktop to mobile apps?

On mobile devices, assistive technologies are part of the operating system, such as VoiceOver on iOS or TalkBack on Android. As soon as they’re activated, the familiar swiping and tapping gestures suddenly have to be performed via keyboard inputs, and the app behaves differently. In the case of a tested mobile patient record app on the iPad, simply rotating the device to landscape mode changed its behavior: individual elements were no longer focused and remained inaccessible to blind users.

Can assistive features for blind people and for people with motor impairments be used simultaneously?

No. Switch control, where an iPhone responds to head movements via its camera and a sound triggers the selection, does not run in parallel with VoiceOver. A blind person would not be able to manage with this combination. The testing criteria do not distinguish between these two functions: they are listed on equal footing and do not specify which type of disability they apply to. The tester must document this limitation themselves.

Is an app accessible if the operating system is the one providing the alternative method of operation?

This is an open question for evaluation. The WCAG requires that an action such as shaking the device can also be performed with a finger, and the software should actually offer this alternative on its own. Through AssistiveTouch, iOS simulates shaking with a finger tap. The task therefore works in this specific context, even though the app itself does not contribute to it. The source of the assistance is irrelevant to the user.

Are testing tools that check for valid HTML and HTML parsing meaningful?

Only to a limited extent. For a person with a disability using the application, HTML validity is often irrelevant because modern browsers have high reliability in interpreting even non-valid markup. With WCAG 2.2, a conformance criterion was removed for the first time, specifically the one regarding HTML parsing. Legally, this will only take effect once the European standard EN 301549 follows suit, which had not yet happened by 2024.

Do accessibility features also benefit people without disabilities?

Yes, this is described by the “curb-cut effect.” A high-contrast or black-and-white mode doesn’t just help people with visual impairments, but anyone looking at a reflective display in sunlight: With pink on green, nothing is visible anymore. Assistive touch is also used in everyday life without people being aware of its original purpose. This is one of the strongest arguments for promoting accessibility.

Is it enough to have people with disabilities participate in testing of the interface?

No, that doesn’t cover everything. A blind person cannot see what they cannot see: If an element isn’t activated at all, they cannot judge whether it’s implemented correctly. A comprehensive test would require people who are hard of hearing, have limited motor skills, are severely visually impaired, and are blind, each with varying degrees of impairment. Some blind people work exclusively with screen readers, while others rely solely on a Braille display.

What causes organizations to fail in providing accessible documents?

It’s the volume, not the technology. The effort required per document is limited, whereas the number of service instructions, forms, and texts within an organization is virtually limitless. Each one would have to be adapted at least once. Partial automation using tools was in the works in 2024, but this does not change the fact that poor content remains poor even when made accessible. Public entities are legally obligated to comply; for online stores in Europe, failure to do so risks penalties from market regulators.

Share this page