Software Test Survey 2024

Agile projects dominate software development with 75% - but the corresponding testing skills are dwindling dramatically. A long-term study over 15 years shows that classic test techniques are becoming massively less important, while tools are swallowing up the methodology. Almost a fifth of testers no longer know statement coverage as a conscious technique, even though the metrics are used on a daily basis. The current survey is the first to systematically examine how AI is changing the testing landscape and whether hybrid process models are a more honest description of reality.

Podcast Episode: Software Test Survey 2024

In this episode, I chatted with Karin Vosseberg, who has been involved with the Software Testing Survey since 2011. We talked about the history and evolution of the survey and how it helps to gain valuable insights into the software testing industry. A big focus was on the shift from traditional to agile projects and the role of AI in the industry. Karin explained that the survey is open to the public and how companies and individuals can benefit from it. I invite you all to take part in the survey and look forward to sharing the results with you.

“What is publicly funded must also be publicly accessible.” - Karin Vosseberg

Prof. Dr.-Ing. Karin Vosseberg has been a university lecturer at Bremerhaven University of Applied Sciences since 2009, specializing in IT system integration and software engineering. She is committed to the integration of software quality assurance, sustainability in software development and digital sovereignty in curricula. She is a strong advocate for gender equality in STEM degree programs and is a member of specialist groups of the German Informatics Society and the Forum Computer Scientists for Peace and Social Responsibility. She has also been a member of the ASQF Executive Committee since 2015.

Highlights der Episode

• Agile projects dominate with 75%, but many use hybrid methods instead of a real agile mindset.
• Test methods such as equivalence partitions or boundary value analysis are becoming less important in practice.
• 18% do not know statement coverage, although dashboards show it daily in agile projects.
• Code-related practices are top, mindset practices such as collective code ownership are used by less than 50%.
• Industry differences in testing practices are minimal - even automotive and medical hardly differ.

Insights into the 2024 software test survey

In this podcast episode, Richie and Karin Vosseberg talk about the Software Testing Survey 2024, which is being conducted for the fourth time. They discuss the genesis of the survey, the evolution of the project landscape and key findings from previous surveys. They also explain how AI and technological innovations are taken into account in the current questions.

Introduction to the 2024 software test survey

Hello and welcome to a new episode of our software testing podcast. I’m Richie, your host, and I’ve prepared a very special episode for you today. We’re looking at the Software Testing Survey 2024, with my esteemed colleague Karin Vosseberg as our guest. Karin has been accompanying this survey since its launch in 2011. The fourth edition is currently running in September, and of course we want to motivate you to take part and take a look at the results so far.

The history of the survey

The idea for the survey originally came from a dissertation in 1997. Together with Andreas Spilner and other colleagues, we decided to launch a long-term study. We wanted to find out how testing in software development has changed over the years. With the support of the German Testing Board and other national boards, we have been conducting this comprehensive survey every four years ever since.

Change from traditional to agile projects

One of the most exciting developments we have observed is the shift from traditional to agile projects. While 75% of projects in 2011 were still phase-oriented and only 25% agile, this ratio has almost reversed by 2020. It is interesting to see that many people continue to maintain traditional role models despite agile working methods. This could be an indication that they define themselves according to their skills rather than their formal roles.

Challenges with agile practices

Although agile methods are becoming increasingly popular, there are some practices that are less widespread than expected. For example, collective code ownership and pair programming are important elements of quality assurance, but are only used by around 50% of respondents. In contrast, program code-related practices are very important. This shows a strong fixation on functioning code, while organizational aspects are often neglected.

Focus on technological innovations

For the current survey, we have also taken into account new technological trends such as artificial intelligence (AI). Among other things, we ask whether AI is used in various test phases - from requirements generation to test case creation. We also investigate whether companies need fewer external resources as a result of using AI. These are exciting questions, the answers to which could provide valuable insights into the current state of the technology.

Benefits of the survey results for testers and companies

The results of our surveys are publicly available and provide valuable insights for both individual testers and companies. They can serve as a benchmark or indicate which skills and technologies could be important in the future. It is particularly interesting for freelancers and the self-employed to see where the market currently stands and which areas offer potential for further training.