Software testing in 2024 will be dominated by three factors: the integration of AI tools into the test process, the EU Accessibility Act as a driver for accessibility testing and the Cyber Resilience Act as a mandatory framework for security testing. AI is not a substitute for human testing experience, but a tool that strengthens good testers and produces bad results faster.
Key Takeaways
- AI-supported test tools are standard today: No significant test tool can do without built-in AI functions for test case creation, GUI automation or log evaluation.
- A good tester delivers better results faster with AI, a bad tester delivers worse results even faster with AI.
- Legislation such as the EU Accessibility Act, the Cyber Resilience Act and the EU AI Act are the strongest drivers for building quality testing earlier into the development process.
- Low-code and no-code test automation enables subject matter experts without programming knowledge to specify and perform tests themselves and will become even more important in 2025.
AI has arrived in testing, the experimental phase is over
Artificial intelligence has evolved from a hype topic to an integral part of test work. Those who called up a language model early in the year, entered a sentence and hoped for complete test cases and test data are now working more soberly. The initial enthusiasm has given way to a critical view, both when testing AI and when testing with AI.
Florian Fieber sums up the situation as follows: “AI was the meta-topic at almost every conference, from dedicated formats such as the SIQ Info Days to Testing United, which dealt exclusively with the topic. The key insight from this year is that all that glitters is not gold when it comes to testing with AI.
Humans remain in the process. AI enhances existing skills, it does not replace them. A good tester delivers better results and works faster with the help of AI. A bad tester produces worse results with AI, just faster.
We can’t take humans out of this, not our skills as testers and our experience. That is essential and that remains. Florian Fieber
How does AI become a tool in daily testing?
AI is becoming a commodity, i.e. a natural tool in the test process. Although every single tester currently uses a language model for their own tasks, this is rarely firmly woven into the testing and development process.
When it comes to testing with AI, the distribution is shifting to the tool landscape. There are hardly any test tools left without built-in AI, be it for test case creation, scripting, developing test ideas, interface recognition in GUI automation or pattern recognition in log files and error logs.
AI assistants are particularly easy to use. APIs and platforms such as OpenAI can be used to build your own assistant for a specific use case. Such an assistant helps, for example, to develop ideas and acceptance criteria from the requirements entered.
It makes sense to create a collection of such assistants that support individual test activities. These tools are gradually integrated in many places.
The spread of AI is heterogeneous, not uniform
AI does not spread homogeneously in testing. The development is not an either-or, and it does not proceed at the same speed across individual test activities, domains or organizations.
Some teams go all-in, other areas are far from it. This pattern is familiar from previous methods and tools that have been introduced over the years.
There is also a disparity within testing activities. AI will become very strong in test execution, test case creation and evaluation. In test planning, monitoring and control, on the other hand, the degree of autonomy will remain lower for a long time.
Why laws are currently the strongest drivers in testing
Legislation is a driver for test topics in the broadest sense. Fixed, tested requirements for products are ideal from a testing perspective because they generate attention and drive the issues forward.
Three regulations are currently acting as such drivers:
| regulation | topic in testing |
|---|---|
| EU Accessibility Act | Accessibility Testing, accessibility of products |
| Cyber Resilience Act | Cyber Security, organizational and procedural evidence |
| EU AI Act | AI Governance, testing and evaluation of AI systems |
Alongside AI, accessibility testing was the second major conference topic of the year, driven by the imminent entry into force of the Accessibility Act. Cyber security and the Cyber Resilience Act also generated a considerable need for clarification, which has not yet been covered.
These requirements are greater than just testing. Cyber security also includes organizational and procedural aspects as well as verification issues that go beyond traditional testing.
Non-functional quality belongs in the design, not at the end
Security, accessibility and similar features must be considered and built in from the outset. They cannot be successfully tested into a finished product at a later stage.
The common reflex is different. First the software is completed, then comes the performance test, the security test or the accessibility test, without having dealt with them in the months beforehand.
The challenge lies in integrating the relevant tests into the pipelines and continuous integration. If you only check accessibility at the end, you will identify open issues and then gradually move the check forward. It makes more sense to focus on requirements engineering right from the start, so that the properties are built in and only need to be verified in the test.
Agility remains undone, and quality is its lever
Many organizations are further away from an end-to-end agile way of working than they realize. Agile transformation, scaling and adoption still have a long way to go, and testing plays a key role in this.
Agility enforces quality. If you work in short iterations and don’t focus on testing and quality, the software will blow up in your face after just a few sprints. This insight was uncomfortable years ago, but today it is a strong driver for testing in the right places.
DevOps and quality engineering are gaining in importance in this environment. Development and testing are becoming more closely interlinked, and testing is moving earlier into the development process.
How low-code and no-code open up testing
Test automation continues to gain in importance, and the low-code and no-code area is on the move. The aim is to decouple technology and functionality.
The principle is not new. Keyword-driven testing has long aimed to abstract from the test object and the underlying technology so that no one has to write their own test scripts.
For you, this means that business users and domain experts can specify and perform tests without being deeply involved in the technology. This empowerment of non-technical people integrates more expertise into test automation.
International standardization thrives on changing perspectives
Standardization in testing makes sense because it creates a common denominator, a basic framework and uniform terms. The market and the participants reflect this need again and again.
Looking beyond national borders changes one’s own perspective. The perception of certification schemes and of the entire testing discipline differs between Germany, Austria, Switzerland and other countries, in some cases considerably. Knowing these perspectives is important, but it takes time and requires compromises.
The work oscillates between two poles. On the one hand, there is the threat of over-regulation and over-standardization, and on the other, the danger of developing something too quickly.
At the ISTQB, a separate working group structures testing in specific domains. The Certified Tester portfolio already includes modules for automotive testing, gaming and gambling, and there is room for further topics. Curricula on DevOps, testing with AI and accessibility testing are likely to follow sooner or later.
