How is agile development changing testing? 

Agility permeates the everyday life of development departments. Where it is not yet practiced, it is being introduced. But what does the change from conventional to agile development approaches look like? What are the pitfalls and how does it change quality assurance, especially testing? What are the consequences for familiar roles in testing? It is obvious that the shift to an agile approach entails profound changes. Test is not spared either. So let's embrace the change and discover the opportunities for testing.

Agile is often promoted as a project management method. This is understandable, because the consulting industry thrives on packaging ideas into sellable and tangible products. Thus, there are many different variants of Agile with different names, which contain a wealth of tool sets, methods and procedures. And those who dutifully implement them should be allowed to experience the benefits: Reducing bureaucracy, accelerating the development of effectively useful results, and maximizing customer value.

In our view, however, agility is much more than that. It is a culture and an attitude that allows a team to pursue their goals in a self-organizing, courageous and transparent way. And through this, profound changes occur: more self-efficacy of the employees, more engagement, greater satisfaction and more flow in the implementation of tasks.

However, changes don't fall from the sky and rarely turn out purely as described in the textbook. You simply have to start with the concrete work somewhere. All the agile principles, methods, best practices, procedures and defined approaches are a very good start.

A start according to Scrum or Kanban à la textbook is a promising entry into the agile world. And also training courses on agile testing, such as the Agile Extension to the ISTQB Certified Tester Foundation Level ¹ or the iSQI Certified Agile Tester ² will get you on your way. And once these processes are up and running, the first successes can be achieved with them. But in our experience, the boost for benefit and success only ignites when the agile values are internalized in the team, or the organization, and the associated processes are established.

One indicator for this are the results of the retrospectives, namely when the team begins to constructively question the lived process and redesign it to fit the concrete situation. The team's own agile approach develops, which fits the team and the context and offers individuals the opportunity to contribute according to their abilities and interests and to achieve the common goals.

Let's pick out a few aspects and questions from agile testing to give you an idea of what to expect on your journey.

Test automation - the test pyramid

Tight and efficient test automation is essential in agile projects. The test pyramid model is often used for this purpose: This describes the types of tests to be performed, their application frequencies, the associated costs, and the dependencies between the individual test types and levels.

For example, it is often the case that automated unit tests form the basis, which are carried out during development and before all other tests. Depending on the structure of the system, various intermediate stages are possible, up to the final manual tests as part of acceptance by the customer (see Figure 1).

The Big Picture: the four test quadrants

The four test quadrants are often mentioned in the same breath as the test pyramid. These often help us in projects to classify all the different types of tests, whether dynamic or static, functional or non-functional, in one picture.

Janet Gregory and Lisa Crispin have given some fundamental thought to which tests are needed in agile projects and who should ideally do them. Basically, test experts like to fall back on the V-model to determine which reviews or tests have to be done by whom and when, even if their project does not work according to the V-model at all. This may still work to some extent for some Kanban teams, since Kanban is basically open here. But in most agile teams, the integration levels are very close together and hardly separated from each other in terms of personnel.

Since Gregory and Crispin both come from an XP environment, they looked for a consistently different approach and thus found the most effective model to date, which has already proven itself many times over in practice ³. They have called it the "four test quadrants" because they have divided the test into four significant groups based on considerations by Brian Marick [Mar03] using two dimensions (see Figure 2).

Test procedure

The ideas of the classical test methods are of course still valid in the agile context and should be applied: the grouping of similar behavior as in equivalence classes, the consideration of borderline cases as in limit value analysis, or the focus on logical and technical states as in state-based testing. At their core, all of these methods continue to provide the ability to create test cases along the lines of: As little as possible, as much as necessary.

However, there is usually not enough time in agile projects to explicitly work through these methods. Therefore, it is essential to internalize the ideas of these methods and not just to work through them procedurally. Of course, this point must be questioned once again in consideration of the use in safety-critical environments.

Exploratory testing

Exploratory testing is a valuable complement to structured testing. In the agile context, these tests take on much more significance. Experienced testing experts such as Cem Kaner and James Bach advocate a creative approach to testing, according to which the tester is driven by his intuition rather than by a method or a tool ⁴. A good, creative tester will guess where the bugs are.

This approach amounts to an exploration of the software. The tester pokes into the system and follows paths that may lead to errors. If he does not find any, he returns and follows another path. Because of his many years of experience, the experienced tester will know where to look. Similar to the term "code smell" common in the development environment, which developers smell when they see suboptimal code, one could say: testers perceive the "bug smell".

In our projects, we have made another exciting observation. Good, experienced and explorative testers use the structured test procedures implicitly - they have already internalized their basic ideas.

And that's just the beginning of the testing process. Session-based testing, AcceptanceTest-Driven Development and Behaviour-Driven Development play with the shared responsibility for quality in the team and help to live quality in the whole process.

Now what about the test manager?

Last but not least, there is the question of roles in the testing process: Do you still need a test manager in an agile project? The answer is quite clear: It depends. It is obvious that in a professional project environment - whether traditional or agile - many quality assurance tasks need to be organized and performed. While in the traditional approach a test manager is the focus of test planning and control, the agile team is supposed to organize the quality itself. But especially in agile projects, it can be enriching for the team to have access to someone who has testing and quality assurance know-how, who knows test procedures, and who can coach the team in testing activities.

The article was published in the 02/2018 issue of German Testing Magazin.