At DB Netz AG, software testing of the future looks like this: quality coaches instead of dedicated testers carry the quality mindset into the agile teams until everyone involved pays attention to quality as a matter of course. The goal is a continuous quality culture. The test-focused CI/CD pipeline MoQ-AP makes testing a fixed part of the toolchain and brings continuous testing into everyday development.
Bettina Buchholz is Strategic Lead for Quality Assurance & Test at DB Netz AG and Product Owner of MoQ-AP (Modular Quality Assurance Pipeline). Andreas Neumeister of CGI is an expert in test data management and test management. Tom Kober of CGI specializes in test automation architectures and test automation strategies. Together, they are reshaping quality assurance at DB Netz AG, with a focus on a sustainable test strategy and continuous testing. In this interview from 2022, they talk about the challenges testing faces, the ideas that could address them, how we will test in the future and how testers and test managers can prepare for it today.
What Challenges Will Software Testing Face in the Future?
Bettina Buchholz: In the past, there were dedicated testers who were responsible for “quality”. Today we no longer talk about testers but about quality coaches, whose job is to get the quality mindset into the agile teams. They help development teams and product owners build an understanding of testing into every stage of the SDLC and find defects early through shift-left. At some point, there will be no more quality coaches and quality engineers, because everyone involved in software development will pay attention to quality assurance and continuous testing without thinking about it. A continuous quality culture will take hold.
In the future, testing will no longer be seen as a burden or extra work, which is still often the case today. It will become a fixed part of the toolchain. Every single development activity will check the quality of the software product, intuitively and automatically.
Andreas Neumeister: Automation will keep rising sharply, so the manual effort we needed until now will be freed up for other important topics. These include security testing, portability testing, accessibility testing, high availability and much more. For test data generation, a mix of automated microservices that understand the requirements and AI-driven data generation will become the norm. That will make a lot of manual work obsolete: collecting data requirements, generating data and manipulating it. At the same time, more effort has to go into planning and building the test data strategy.
Tom Kober: Given the current global situation, security vulnerabilities matter more than ever. Testing has to deal much more closely with security, so that weaknesses are avoided early and as an integrated part of the work. New technologies and processes also need new testing approaches, so testing serverless services, AI and machine learning, for example, will move more and more to the forefront. That means developing test methods for these technologies, but also using the opportunities they open up for software testing in general.
A major factor in designing a test automation architecture is the effort it takes to maintain and extend automated tests. Self-healing tests and supporting test design tools already address this. Most of them, however, are not yet mature enough to produce sustainable, reusable tests.
What Ideas Could Build a Continuous Quality Culture?
Tom Kober: Integrated, standardized scanning tools make proactive defect prevention possible. Through continuous integration, they raise awareness of software quality and security even before a test environment exists. That way, useful information about software quality can be shared among stakeholders, and improvements can be made early. At the same time, compliance checks and standards can be defined for each organization with individual rule sets.
Smart analysis options and the views that come with them lead to better assumptions and more precise tests, which improves quality assurance. Testing effort can then be targeted better, which can mean faster test feedback or lower costs, among other things.
Bettina Buchholz: Machine learning will become an increasingly relevant part of software development. Today, testing is still seen as extra work. Machine learning will increase acceptance of software quality and testing activities and ideally make them a matter of course.
Test processes, with all their activities, have to be integrated, for example into CI/CD pipelines, and cover all test levels and test types, which leads to almost complete automation.
The biggest challenge, though, is changing the mindset and establishing a continuous quality culture. Today we think too much in terms of “shift-left and shift-right”. We need to get to “quality overall”.
Andreas Neumeister: In line with the quality culture Bettina describes, development, security, operations and quality assurance will become more closely intertwined. That will develop the skill sets of the people in the projects and also move the technical practices in these areas forward. With automation increasing and AI technologies coming in, a clean setup of test strategies, including automation and data, will matter more and more. This helps especially with comprehensive integration tests across heterogeneous IT environments. Companies and technologies keep evolving, so the ability to test continuously, across technologies and systems, is becoming more and more important.
How Will We Test in the Future?
Tom Kober: Open source tools are spreading fast in combination with container-based test applications, and they form a cornerstone of tomorrow’s test infrastructure. This makes test automation tools sustainable to use and cuts the upfront effort of building the test infrastructure. Local test tools are only used for developing tests, and static test environments only for manual tests and for developing automated tests. With this high degree of automation, reusability needs particular attention, so that the effort pays for itself.
Bettina Buchholz: I think we are well on the way to the future of testing. Test execution has become much faster, but the feedback cycles are still far too long.
If we look toward machine learning and AI, completely new possibilities for testing open up.
Test cases and test suites have to become more maintainable or, ideally, maintain themselves. AI will make it possible for test cases to learn on their own whether they are needed, in what form, and what the user or the system under test needs.
Experience-based testing also matters for a risk-based approach, and here too AI will support us much more in the future. Test data will be reproduced based on experience from earlier test runs, for example, or test cases will be designed automatically from analyses of earlier defect clusters. Test automation will also detect and fix faults in its own system, removing sources of error from within. Quality assurance becomes a constant companion that people experience as support in software development.
Andreas Neumeister: On top of that, future AI quality assurance systems will analyze user behavior in existing systems. They will find defects and weak spots, repair them on their own and then test them automatically against basic quality assurance guidelines and normal user behavior. Good monitoring moves more of quality assurance into operations. To make this possible and cut time to market for new products and features, sustainable analysis and monitoring functions have to be designed and put in place. Knowing how a system works and what state it is in in production is fundamental for quality assurance in operations.
How Can Testers and Test Managers Prepare Today?
Andreas Neumeister: Testers and test managers should reflect on their test approach regularly, ideally with the whole development team, and act on the improvements they find. They should also talk to colleagues from other projects and read success stories from other quality assurance projects. And they should keep learning in a world that changes fast. The quality assurance approach of five years ago is already out of date. Automation will only become more important, so test managers and testers should get a broad understanding of the topic now. It is not about being able to implement automation technically. It is about spotting automation potential early, so you can set up an automation strategy for your own project.
Tom Kober: It is not enough to share an understanding of testing within the team. Testers and test managers also need to be involved in the processes and technologies used in software development and in the projects. That encourages exchange between stakeholders, so awareness of software quality gets passed on and sinks in. This is the first step toward a shared understanding of software quality. Only then can today’s state-of-the-art technologies combine with proven testing practices to create new innovations in software testing.
There is already an expectation that test automation engineers are responsible not only for their tests but also for automated test execution and for integrating their tests into the SDLC. That is why they need to get into topics such as DevOps, CI/CD and cloud platforms and build an overall understanding of development, operations and testing.
Bettina Buchholz: For me, the first step is not to be afraid of new things: to try them and to accept that we might fail. Mistakes are how we get better.
Right now, our main task should be to help the team members, product owners and project managers of this world understand quality assurance, and as coaches to support the shift in mindset toward a quality culture.
As Tom Kober already said: on the technical side, we testers need to get much more involved with CI/CD and continuous testing. We need to find a way to make testing an integral part of the SDLC. Only then will it be easier for team members who are not testers to get into the topic and develop a tester mindset. That lays the foundation for a future quality culture.
Frequently Asked Questions
What Sets a Quality Coach Apart from a Traditional Tester?
A quality coach instills a quality mindset within agile teams, rather than being solely responsible for quality. In the past, there were dedicated testers who were responsible for quality. Coaches help development teams and product owners embed a testing mindset at every stage of the SDLC and detect defects early by shifting left. In the long term, the role is intended to become obsolete because all stakeholders will proactively prioritize quality.
How do testing teams benefit as the level of automation increases?
The manual effort freed up can be redirected to areas that were previously neglected: security testing, portability testing, accessibility testing, and high-availability testing. The benefit, therefore, lies not only in faster test runs but also in broader coverage of quality characteristics. Reusability is a prerequisite: maintaining and developing automated tests requires effort that would otherwise not pay for itself.
Does the planning effort decrease when test data is generated automatically or using AI?
No. In 2022, a mix of requirement-aware microservices and AI-driven data generation was expected to eliminate the need for manual work in capturing data requirements, as well as in generating and manipulating data. In return, more effort must be devoted to planning and establishing the test data strategy. The focus of work shifts from execution to design.
How mature are self-healing tests?
They alleviated some of the burden of maintaining automated tests, but did not solve the problem in 2022. At that time, self-healing tests and supporting test design tools were generally not yet advanced enough to produce sustainable and reusable tests. The effort required for maintaining and further developing the tests remained a major factor in the design of a test automation architecture.
Is it enough to consistently shift testing forward?
No. “Shift Left” and “Shift Right” remain directional thinking; the goal is “Quality Overall”: Test processes, including all activities, are integrated into CI/CD pipelines and cover all test levels and test types, extending to near-complete automation. The biggest hurdle here is not the technology, but the shift in mindset toward a continuous quality culture.
Why is quality assurance shifting more toward operations?
Because knowledge of a system’s function and state in production forms the foundation of quality assurance in the operations domain. The expectation was for AI-powered systems that analyze user behavior in existing systems, identify weaknesses, repair them autonomously, and then automatically retest in accordance with quality assurance guidelines and typical user behavior. To achieve this, robust analysis and monitoring functions must be designed.
Do test automation engineers need to be familiar with DevOps and the cloud?
Yes. The requirements go beyond simply writing tests: automation engineers are also responsible for the automated execution and integration of their tests into the SDLC. This requires knowledge of DevOps, CI/CD, and cloud platforms. This fosters a shared understanding of development, operations, and testing rather than separate areas of responsibility.
How can test managers prepare for greater automation without writing code themselves?
By identifying opportunities for automation early on and using them to develop an automation strategy for their project. Technical implementation is not their job. It’s also helpful to regularly reflect on the testing process with the entire development team, exchange ideas with colleagues from other projects, and engage in ongoing professional development. Approaches become outdated quickly: a five-year-old quality assurance approach is rarely still relevant.
