The 2024 Software Testing Survey is an industry-wide study of how software testing is done in companies in Germany, Austria and Switzerland, run every four to five years since 2011. More than 800 people from development, testing and management took part. Its key findings: a third of respondents already use AI for coding and for quality assurance of that code, while basic test techniques such as equivalence partitioning, and security testing, are used far too rarely.
Key Takeaways
- A third of the testers surveyed already use AI for coding and for checking the quality of that code. For programming alone, the figure among hands-on practitioners is almost 50 percent.
- Only around 40 percent of respondents use test techniques such as equivalence partitioning and boundary value analysis, even though ISTQB certification has taught these basics for more than 20 years.
- 12 percent of participants still don’t automate regression testing at all, and a quarter don’t automate load and performance testing either.
- Test-driven development has been losing ground for years, from 67 percent in 2016 to 48 percent in 2024, even as agile process models become more common.
- Two thirds of hands-on practitioners see a greater need for training in IT security while also rating themselves as well prepared, which points to a clear gap in self-perception.
A Software Testing Survey That Tracks Trends Since 2011
The German Testing Board’s software testing survey captures the state of testing in German-speaking countries: what teams actually do, where they struggle and how things have shifted over the years. It has run at regular intervals since 2011, with editions in 2011, 2016, 2020 and 2024, so it now covers almost 15 years.
The 2024 edition is based on more than 800 participants. That is a statistically sound sample, although deeper analysis hits its limits in places. When many individual questions are cross-tabulated with further categories, the number of answers in each subgroup shrinks, and the data shows trends rather than hard conclusions.
The analysis has academic backing. This is not a quick LinkedIn poll: it rests on null hypotheses, correlations and a technical report running to several hundred pages. The results come in three formats for different readers: a management summary, a brochure and the full technical report.
Why Three Questionnaires Reveal More Than One
The survey splits respondents by role, and that split is what makes many of the results meaningful. There is one questionnaire for quality assurance managers, one for hands-on practitioners, and one for research. That makes it possible to put the different perspectives side by side.
Outsourcing is a good example. Managers tend to want to hand off as much as possible. Practitioners are more selective: they would rather outsource technical work such as test automation, but they are wary of handing softer disciplines such as architecture or security management to outside parties.
Participants come from a wide range of industries and company sizes. The largest groups are in financial services, the public sector and automotive, in line with figures from Bitkom, the German digital industry association. Respondents include corporations with more than a thousand employees as well as small firms with 10 to 20 people.
How Widely Teams Use AI for Coding and Testing
AI was part of the survey for the first time in 2024, and the result is clear: a third of participants already use AI for coding and for quality assurance of that code. That is remarkable reach, given that generative tools only became widely available about two and a half years earlier.
The numbers are highest for programming itself. Almost 50 percent of practitioners already use AI there or plan to in the near future. For testing tasks, the figures are much lower.
A rule of thumb applies: the more technical the task, the more people trust AI with it. For design decisions, project management and test management, where human factors matter more, the numbers drop significantly.
That has consequences for career planning. Pure development work is likely to shrink as a stand-alone profile. People who take on an architect’s role, making components work together and negotiating the interfaces between people, are investing in the right direction.
“If you work with AI and don’t know what white-box testing is, don’t know what an equivalence partition is, how am I supposed to judge whether what the AI generates for me makes any sense?”
(Frank Simon)
Test Automation Has Stalled Since 2020
Test automation has barely moved in years, and that is the sore spot of this survey. Automation grew from 2011 to 2020 and has stalled since then. It is still concentrated in the lower test levels.
The regression testing numbers are sobering. 12 percent of participants say they don’t automate regression testing at all, even though it is the obvious candidate for automation.
Load and performance testing looks even worse. A quarter of respondents don’t automate it. It’s hard to imagine running a load test by hand, so a perception problem is probably at play. In agile projects with a CI/CD pipeline, load and performance often don’t show up in daily work. They are handled by teams further up and never reach the individual tester’s view.
End-to-end test automation is a big topic at conferences and in companies. The numbers don’t reflect that. Very little is actually happening there.
Security Testing: Self-Image and Reality Drift Apart
The biggest gap shows up in security testing and in load and performance testing. Testers consider their work in these areas effective, yet they also know that customers do end up with security problems. That gap can’t last.
Security testing is one of the least reported test types overall. That ties in with a lack of basic knowledge: 12 percent of respondents say they don’t know what a white-box test is. Real security testing is technically demanding, and anyone who doesn’t know what white-box testing involves is probably not doing much of it.
At least many people see their own gap. 66 percent of practitioners report a greater need for training in IT security, and a third in load and performance testing. That self-assessment is accurate: before you can judge what tools and AI produce, you need solid knowledge of the subject.
Test Design Techniques Are Fading from Practice
Classic test design techniques are being used less, not more. Boundary value analysis and equivalence partitioning sit at just under 40 percent, even though equivalence partitioning is one of the most intuitive techniques there is. There has been no progress.
That is surprising after more than 20 years of certification by ISTQB and the German Testing Board. With so many Certified Tester courses delivered, knowledge of the techniques should be much more widespread. Instead, the conversation is almost entirely about automation and hardly ever about how to narrow down the test cases worth automating in the first place.
More tools won’t fix this on their own. Modern tools have provided white-box information and coverage figures for a long time. If users still say they don’t know the concept, there is little reason to think AI will help.
Test management shows a similar picture. Only just under 50 percent of respondents use a test management tool at all. The rest probably work in Excel. A tool is no substitute for understanding, but for a discipline like quality, that rate is thin.
Agile or Hybrid? Many Teams Can’t Decide
In 2024, hybrid is as common as agile, and that exposes an inconsistency. About as many participants call their approach agile as call it hybrid. Hybrid often hides an inability to commit: people work in a supposedly agile way while the executive floor keeps making the key decisions.
The distribution of roles tells the same story. Most respondents place themselves in the classic roles of developer or tester. Genuinely agile roles such as Scrum Master, Product Owner or agile team member each come in below 5 percent. And some of those who call themselves agile team members still work in a classic, plan-driven process model.
Test-driven development is especially telling, because it shows a clear downward trend:
| Year | Use of Test-Driven Development |
|---|---|
| 2016 | 67 % |
| 2020 | 58 % |
| 2024 | 48 % |
For anyone who values agile practices, the trend is heading the wrong way. Pair programming, collective code ownership and stand-up meetings as a quality measure are all below 50 percent. Practices close to development are common, while the practices that change mindset and organization are falling behind.
Software Testing Trends 2024: What Testers and Test Managers Should Take Away
Certificates still matter for careers. In 2024, a third of managers again say that certification is mandatory and non-negotiable. A certificate helps a career, it doesn’t hold it back.
Security testing deserves active advocacy within your own teams. A security flaw rarely gives you a second chance. Yet this is exactly the discipline the survey shows people dismissing as not very exciting, which is hard to understand given the real risks.
The certification courses themselves could be rethought. A four-day course teaches the vocabulary, but not how to apply it in your own context. Support afterwards would make sense, for example through mentoring or user groups where people can bring a specific problem and talk it through. That way, techniques are not just learned as terms but actually absorbed.
Despite faster machines and better automation, being able to make a statement about the quality of testing remains the real achievement. People who master the techniques can judge what tools and AI actually deliver.
Where to Find the 2024 Software Testing Survey Results
All results are available free of charge and without registration at softwaretest-umfrage.de. The site already has the management summary and the statistical breakdown of every question with charts, along with the results of earlier surveys.
The detailed brochure with the best-of analyses runs to around 50 to 60 pages and was scheduled for the end of May 2025. The full technical report was expected by the end of summer 2025, much earlier than in previous years. Talks and shorter reports accompany the publication, including at the German Testing Day and in SQ Magazin.
Frequently Asked Questions
How reliable are the figures from an industry-wide testing survey like this one?
The 2024 survey is based on responses from over 800 participants in German-speaking countries and is backed by scientific methodology, including null hypotheses, correlations, and a technical report spanning several hundred pages. Limitations become apparent in the in-depth analysis: When many individual questions are cross-tabulated with additional categories, the number of responses per intersection decreases. In such cases, it’s easier to identify trends than to make definitive statements.
Do managers and operational staff differ in terms of which testing tasks they would outsource?
Yes, and quite significantly. Quality assurance managers tend to outsource as much as possible. Operational staff are more selective: they are more likely to outsource technical tasks such as test automation, but are critical of outsourcing softer disciplines such as architecture or security management. The difference becomes apparent only because management and operational roles are surveyed separately.
For which tasks are development and testing teams already using AI?
Programming is where AI is most widely used: Nearly 50 percent of operational staff use AI there or plan to do so in the near future. One-third use it for coding and quality assurance; for testing tasks, the figures are noticeably lower. The more technical the task, the more AI is trusted to handle it. For design decisions, project management, and test management, the figures drop significantly.
Why do teams say they do not automate load and performance tests?
A quarter of respondents give this reason, even though it’s hard to imagine load testing being performed manually. This is likely due to a perception issue: In agile projects with CI/CD pipelines, load and performance are often handled by teams further up and do not appear in the day-to-day work of individual testers. When it comes to regression testing, 12 percent do not use automation at all.
What causes security testing to fail in practice?
A lack of basic knowledge. Twelve percent of respondents do not know what a white-box test is, and a true security test is technically demanding. At the same time, testers consider their work in this area to be effective, even though customers report real security issues. After all, 66 percent of those working in operations cite an increased need for further training in IT security.
Does AI make traditional testing knowledge obsolete?
No. Anyone who doesn’t know what a white-box test or an equivalence partition is cannot assess whether generated test cases make sense. Modern tools have long provided white-box information and coverage metrics anyway; if users don’t understand the concept behind them, even AI won’t change that. The assessment of testing quality remains the true measure of performance.
How widespread are traditional test design methods such as boundary value analysis and equivalence partitioning?
Just under 40 percent of respondents use boundary value analysis and equivalence partitioning, and no progress is evident compared to earlier surveys. This is surprising after more than 20 years of certification by ISTQB and the German Testing Board. The discussion revolves almost exclusively around automation, with little focus on how to sensibly narrow down the test cases to be automated in the first place.
Why is test-driven development declining even as agile methodologies are on the rise?
The use of test-driven development fell from 67 percent in 2016 to 58 percent in 2020 and 48 percent in 2024. Pair programming, collective code ownership, and stand-ups as quality assurance measures are also below 50 percent. Many teams describe their approach as hybrid and claim to work in an agile manner, while key decisions continue to be made at the executive level.
