In the digital transformation, large systems are turning into complex ones, and software testing and quality assurance have to grow with them. Test automation, agile processes and microservices keep even complex projects testable, backed by a quality mindset across the whole team.
Who still remembers how our grandparents worked? What will our grandchildren learn about the way we work? Will they wonder how people ever worked in assigned offices, and why collaborating across continents was such a challenge?
The theme behind these questions is the digital transformation toward Industry 4.0. It is about quality assurance in internationally connected companies that operate across all borders, and about what they demand from modern development projects: faster, more agile and with usable intermediate results. How good does the quality have to be, and at what point? How do we make sure quality assurance scales with the complexity of a project? And how do the recurring questions about test efficiency fit with the architecture patterns in use or with development approaches such as agile?
Impressions from Silicon Valley
Richie has just come back from Silicon Valley, packed with new ideas, inspiration and humility in the face of what is developing there right now. We talked at length about what he saw at the start-ups and companies there, and about the challenges that new technologies, ways of working and systems pose for software testing and quality.
Given the changes ahead for many traditional industries, the word “digitalization” sounds almost cute. The changes will probably be disruptive, opening up new lines of business and making others disappear. Some industries still cling to the false hope that this cup will pass them by. Some see only concerns and problems while they are already being overtaken on the left and right. And others see the opportunity and use the potential that software and data already offer today. Whether those last ones are working on artificial intelligence, autonomous systems, predictive techniques or augmented reality, our core topic, software quality, is deeply affected.
How the Digital Transformation Turns Large Systems into Complex Ones
The past few decades were shaped by systems getting bigger and bigger. Processes, methods and tools were created to keep them under control. Ongoing optimization (test centers, for example), improvements (more efficient workflows, better tools) and more automation (test automation at every test level, for instance) helped us deliver efficient quality assurance even for large systems.
But large systems are being replaced by complex ones. Their built-in dependencies, relationships and algorithms are becoming harder and harder to oversee and understand. Yet they still need quality assurance, meaning at least verification and usually testing as well. We see more and more ways of adapting to this shift. Process models are designed so that fast, efficient quality assurance is still possible. System architectures are designed so that encapsulated units can be understood and focused quality assurance remains feasible.
Processes and Architecture Make Testing Efficient
The usual way to increase efficiency is automation. For testing, the first step is automated test execution. The benefits are obvious: fast execution and fast feedback, especially for regression testing. Only preparing the test scripts takes significant effort, both when creating them and when adapting them.
But how do we apply this in environments that keep changing? Agile development processes involve frequent changes. Every sprint needs automatically executable test scripts that evaluate the functional quality of the system. Because changes in the current sprint are likely, those scripts would also have to be adapted quickly. For any nontrivial project that is prone to change, adapting a significant share of the test scripts by hand every sprint can only be done at high cost.
So should we give up on automated test execution? Not at all! Instead, we need to find ways to reduce the effort of adapting the test scripts. Proven approaches include data-driven, keyword-driven and model-based test design techniques. On top of that, quality assurance is no longer just the tester’s job at the end of the process chain. Implementing the necessary tests is part of development, just like writing and checking in the source code itself.
The next interesting question is how to keep individual tests focused. How can we reduce the influence of other functions on the function actually under test? This is where architecture plays an important role. Small, replaceable units are used heavily in the microservices architecture approach: every microservice has its own tests, every microservice is independent of the others and communicates only through defined interfaces, and every microservice focuses on a specific customer benefit.
Small teams, a clear architecture, focused tests and customer benefit front and center: all of these advantages can come from a smart choice of architectures, processes and test approaches. We have highlighted microservices, agile and test automation as examples. Combined, they keep even complex projects maintainable, testable and understandable. Test effort stays reasonable, the test focus stays on customer benefit, and quality assurance scales with the complexity of the system being built.
Quality as an Attitude in the Team
Alongside these technical factors, the soft factors are invaluable. Anyone who wants to build excellent software today thinks about the development process as a whole: besides methods and tools, above all people and mindset. When everything comes together in a flow, people can realize their potential and innovation happens. Quality is ensured by the team. Quality assurance becomes a shared task, carried by the attitude of the whole team.
For many team members, that means a massive change process that takes experience, empathy, dedication and patience. A workshop or training course won’t do it. It takes ongoing support, regular impulses and guidance so that a team can find its own way to live agile values such as courage, transparency, openness and self-organization. The team members’ attitude toward the project and toward the chosen process makes a decisive difference to whether any project succeeds or fails!
Of course, solid training is the foundation such a change process has to rest on: speaking the same “language,” mastering the tools of the quality trade, understanding and applying the methods and approaches. For software testing, the Certified Tester scheme provides exactly that, as the world’s leading professional training scheme for software quality.
Besides the factors that support the success of a development project, there are many more that decide whether the resulting product succeeds.
Usability: Don’t Make Me …
Whether people accept a product, an app or software depends more and more on usability and on what users experience. And that no longer applies only to consumers. Expectations of business software are rising too: when employees have to wrestle with slow, cluttered and complicated interfaces for hours every day, frustration is guaranteed. Start-ups target these pain points and solve them. Three broad guidelines come up again and again, and they are worth following:
- Don’t make me wait: Nobody wants to wait for anything anymore. Real-time availability is what counts. Loading bars, standing in line and waiting are deal-breakers.
- Don’t make me ask: Every question slows things down. The system knows me and has my data. Asking again is a no-go!
- Don’t make me think: Take the thinking off my hands! Anyone who sits in front of an interface or website and first has to figure out how what works where and why will soon be tempted to click the “x,” or at least won’t enjoy using it much. We are spoiled by apps and the like. You clicked here? Then you might also like this and that …
These aspects rarely concern single components. They usually affect the user’s entire workflow. Is it even a tester’s place to question the business processes? Absolutely! The tester’s critical eye is exactly what’s needed here. How can the application be made simpler? What can be taken off the user’s plate? Do we still need features X and Y? Which old habits (interfaces, special cases, modules, for example) can be cut? What has “grown historically” and could be rethought?
What Comes Next?
So what do we do with the impressions from Richie’s trip? We live in exciting times. Technology, and today’s software in particular, will give us possibilities in the coming years that we probably can’t imagine yet, or don’t want to believe.
Autonomous driving is a good example. What is often met with extensive concerns and regulations here has reached very different dimensions in other countries. Walk through the streets of San Francisco today, and every few minutes you’ll see a test vehicle from one of the countless companies allowed to drive on public roads. Progress in this technology has been enormous in recent years. The demands on quality and quality assurance in this field are huge and are unlikely to shrink. And accidents like the one involving Uber make international headlines immediately.
Against this background, it is easy to forget that traffic with humans at the wheel causes numerous fatal accidents every day. A direct comparison also shows that humans are often to blame in accidents involving self-driving cars. The challenge, of course, is and remains to reduce the number of accidents to a minimum, and that not a single one may be caused by an error of a self-driving vehicle.
That is the challenge we have to face as testers and people responsible for quality: with processes, architecture designs, tools, training and the right mindset.
Frequently Asked Questions
What Does the Shift from Large to Complex Systems Mean for Quality Assurance?
Large systems could be managed using processes, test centers, more efficient workflows, and test automation at all test levels. Complex systems are different: their dependencies, relationships, and algorithms are difficult to grasp. Nevertheless, they still need to be verified and, in most cases, subjected to testing. The solution lies in process models and architectures that keep encapsulated units manageable and allow for focused quality assurance.
Why does test automation quickly become expensive in agile projects?
It’s not the execution that costs money, but the maintenance. The system changes in every sprint, and a significant portion of the test scripts must be updated accordingly. Manual adaptation in non-trivial, change-prone projects is feasible only at a high cost. That’s why you shouldn’t give up on automated execution. Instead, reduce the effort required for adaptation: data-driven, keyword-driven, or model-based.
What advantage does a microservice architecture offer for testing?
It keeps tests focused. Each microservice comes with its own tests, is independent of the others, communicates only via defined interfaces, and targets a specific customer benefit. This reduces the influence of external functions on the function that is actually being tested. Combined with small teams and an agile approach, even complex projects remain maintainable and have high testability.
Is quality assurance the responsibility of a separate testing team at the end of development?
No. Quality is ensured by the entire team, and implementing the necessary tests is just as much a part of development as writing and checking in the source code itself. In addition to methods and tools, people and mindset matter. The team members’ attitude toward the project and the chosen process plays a decisive role in determining success or failure.
Is training alone enough to transition a team to agile quality responsibility?
No, a workshop or training session alone isn’t enough. The transformation requires experience, empathy, dedication, and patience, as well as ongoing support, constant encouragement, and guidance so that a team can find its own path to courage, transparency, openness, and self-organization. A solid foundation in training remains essential for this.
How can you tell if an application isn’t unnecessarily holding users back?
By following three guidelines. “Don’t make me wait”: Loading bars, waiting in lines, and delays are deal-breakers; real-time availability is what’s needed. Don’t make me ask: A system that already has the data shouldn’t ask for it again. Don’t make me think: If users have to think about how an interface works, they’ll quickly click away. This also applies to business software.
Are testers allowed to question the business processes behind an application?
Yes, the tester’s critical eye is especially valuable here. Usability rarely concerns individual components, but rather the user’s entire workflow. Useful questions include: How can the application be made simpler? What tasks can be taken off the user’s plate? Are certain features still necessary? Which legacy interfaces, special features, or modules can be eliminated?
