Agile Testing - Problems and Success Factors 

 29 March 2021

The agile transformation brings challenges with it. After many software testing projects that we have accompanied, there are nevertheless a few typical problems that occur again and again, despite all the individuality. And there are also a few factors that positively influence and accelerate the transformation.

Problem #1: Limitation to the test

Agile procedures create transparency. And so deficiencies in quality quickly become visible. The first reaction in practice is often to focus the concentrated energy on "agile testing" or "agile testers" - precisely with the motivation to focus on quality. However, this approach only partially solves the problems.

Quality in the agile team means above all responsibility of each individual. And even though it makes a lot of sense to have a tester's perspective in the team, that doesn't exempt them from the shared responsibility for quality. It is not the agile tester who brings quality, nor the testing methods. Quality must be lived in the entire team, by the developer, product owner, tester, and whatever other roles you want to fill.

Therefore, when dealing with testing in an agile context, it is important to always keep in mind that it is only one part of the quality measures. It is much more important to work on promoting and living quality in the team. Thus, all of the following methods and tips provide an impulse and starting point, which, however, must be thought through further in the projects individually and depending on the context.

Problem #2: no patience in agile testing

Agility is change and change takes time. Transferring testing with its various aspects into the agile context takes time. In addition, agile is always about questioning the existing - this also applies to test methods and test approaches. Here it is important to be vigilant and not to judge hastily whether a change has made a difference or not. The effects of a change may only be visible after some time or iterations. Those who hastily reverse the change may miss a valuable opportunity for further development.

Problem #3: THIS is the solution

Tools, methods or gurus are often seen as the solution to all the challenges presented by the transition to agile. Typical example "to speed up" testing: test automation. But if the foundation is not stable, even the tool does not solve the problem. Quite the opposite: "If you automate crap, you still have crap - only faster". So it's also easy to fall into the method trap: We have to test this way! Why? Because it says so in the book! Likewise, the saying "A fool with a tool is still a fool" reveals that the use of tools (e.g. for test automation) needs to be well considered and chosen to fit the rest of the project.

Success factor #1: Freedom for personal responsibility

Agility thrives on values such as self-responsibility, transparency, and courageous decisions. For this, the necessary freedom must also be left for the development of the test strategy, for example. Testers must have the opportunity to try out methods and tools, discard them again, and take other paths. This also requires an appropriate error culture, in which errors are seen as an opportunity for learning in the team and not as a failure of an individual. This is particularly important when moderating retrospectives.

A change in thinking by superiors, e.g. the line test manager, is also necessary here. If the testers' decisions are repeatedly questioned or overruled from above, this quickly affects the motivation in the team and leads to the exact opposite of what one wants to achieve with the agile approach: Instead of a team bubbling over with creativity and performance, the focus is then on things like safeguarding, logging and decision templates.

Success factor #2: Soften classical thinking

Sometimes the biggest challenge in the transformation is to get the "old" thinking out of people's heads. This applies to the team as a whole, but especially also to testers who come from traditional projects into agile. For the tester, it is no longer about working through "his" area - test concept - test design - test execution - defect management, but about making his know-how available to the team. Similarly, it is often unusual for developers from a traditional project environment to contribute to quality themselves and not just throw the finished modules "over the wall" to the test team.

This breaking down of silo thinking requires a sure instinct when it comes to support. It is a balancing act between empathy and maintaining the necessary pressure to change. It is also a process that can take several months until these behavior patterns are changed.

Success factor #3: Retrospectives

In our experience, one of the biggest levers for the success of agile testing is the proper execution of retrospectives. By the way, this does not only apply to the agile context, but also to the traditional one. The emphasis here is on "properly". Because a meeting in which the majority is not active and the rest is "complaining about the last few weeks" has nothing to do with a retrospective. It is precisely in this self-reflection of the team that the power lies to quickly assess the paths taken, to correct processes and to address the elephant in the room - about which no one else is talking.

For quality and test topics, it is worthwhile to conduct separate retrospectives that address only these topics, if required. This can be particularly helpful in the start-up phase of agile testing, when the basic test strategy or the characteristics of test automation are being developed. Nothing is more frustrating than realizing after weeks or months that the path taken - e.g. in the choice of test tools or a framework - was the wrong one. Retrospectives can help to uncover these points early on and prevent conflicts from arising in the first place.

Bonus success factor #4: Your personal path

The concrete implementation of agile testing depends on many factors: Company, project, structures, application, team.... This affects everything from release planning and the proportion of hardening iterations to iterations, to the type and extent of customer feedback, to the degree of test automation, to the level of detail and scope of results, and their documentation. Every project and every organization is highly individual and the shift to agile methods thrives on experience, best practices and sharing. And so we would like to encourage you here: talk to others about their projects. There is so much knowledge and experience in your company and networks - use it and shape your own agile path.