Agile development changes testing. The test pyramid organizes automation, and the agile testing quadrants place every type of test in a single picture. Classic test techniques and exploratory testing remain important, and whether an agile team still needs a test manager depends on the context.
Agility runs through the daily work of development departments. Where it isn’t practiced yet, it’s being introduced right now. But what does the move from traditional to agile development look like? What are the pitfalls, and how does it change quality assurance, and testing in particular? What happens to the familiar roles in testing? The switch to an agile way of working clearly brings deep changes, and testing is no exception. So let’s embrace the change and look at what it opens up for testing.
Agile Is More than a Project Management Method
Agile is often marketed as a project management method. That’s understandable: the consulting industry makes its living by packaging ideas into tangible products it can sell. So there are all kinds of agile flavors under different names, each with a wealth of toolsets, methods and procedures. Follow them faithfully, the promise goes, and you’ll see the benefits: less bureaucracy, faster delivery of results that are actually useful, and maximum value for the customer.
In our view, agility is much more than that. It is a culture and a mindset that lets a team pursue its goals in a self-organized, courageous and transparent way. That leads to deep changes: employees who feel more effective, more engagement, greater satisfaction and more flow in getting the work done.
Change doesn’t fall from the sky, though, and it rarely goes exactly the way the textbook describes. At some point you simply have to start doing the actual work. All the agile principles, methods, best practices, procedures and defined approaches are a very good place to begin.
Starting with Scrum or Kanban by the book is a promising way into the agile world. Training in agile testing, such as the Agile Tester extension to the ISTQB Certified Tester Foundation Level or the iSQI Certified Agile Tester, will also get you going. Once these processes are running, the first successes follow. In our experience, though, the real boost in value and success only kicks in once the team, or the organization, has internalized the agile values and the related processes are established.
The results of retrospectives are one indicator: the moment the team starts to question its own process constructively and reshape it for the situation at hand. The team develops its own agile approach, one that fits the team and the context and lets each person contribute according to their skills and interests while working toward shared goals.
Here we pick out a few aspects of and questions about agile testing to give you an idea of what to expect on your journey.
Test Automation: The Test Pyramid
Agile projects can’t do without lean, efficient test automation. The test pyramid is a common model for it. It describes the types of tests to run, how often each is used, what they cost and how the individual test types and test levels depend on each other.
Often, for example, automated unit tests form the base. They run alongside development and before all other tests. Depending on how the system is built, various intermediate levels are possible, all the way up to the final manual tests during customer acceptance.
The Big Picture: The Four Agile Testing Quadrants
The four testing quadrants are often mentioned in the same breath as the test pyramid. In our projects they help us place all the different test types, dynamic or static, functional or non-functional, in one picture.
Janet Gregory and Lisa Crispin once thought through, from the ground up, which tests agile projects need and who should ideally do them. Testing experts like to fall back on the V-model to decide which reviews or tests someone has to do and when, even if their project doesn’t follow the V-model at all. That may still work reasonably well for some Kanban teams, because Kanban is basically open on this point. In most agile teams, however, the test levels sit very close together and are hardly separated in terms of who does the work.
Because Gregory and Crispin both come from an XP background, they looked for a radically different approach and found what is still the most effective model, one that has proven itself many times in practice. They called it the “four testing quadrants,” because, building on ideas from Brian Marick, they divided testing into four meaningful groups along two dimensions.
One axis asks whether a test is business-facing or technology-facing. The other asks whether it supports the team during development or critiques the product. That gives four quadrants:
- Q1, technology-facing and supporting the team: unit and component tests, mostly automated and written by the developers.
- Q2, business-facing and supporting the team: functional tests, examples and story tests, often automated, plus prototypes and simulations.
- Q3, business-facing and critiquing the product: exploratory testing, scenarios, usability testing and acceptance testing, mostly manual.
- Q4, technology-facing and critiquing the product: performance, load and security testing, usually with tools.
Test Techniques
The ideas behind the classic test techniques are, of course, still valid in an agile context and should be applied: grouping similar behavior, as in equivalence partitioning, looking at edge cases, as in boundary value analysis, or focusing on logical and technical states, as in state transition testing. At their core, all of these methods still let you design test cases by the motto: as few as possible, as many as necessary.
In agile projects, though, there usually isn’t time to work through these techniques explicitly. That makes it all the more important to internalize their ideas instead of just following them as a procedure. In safety-critical environments, of course, this point needs to be reconsidered.
Exploratory Testing
Exploratory testing is a valuable complement to structured tests, and in an agile context it becomes even more important. Experienced testing experts such as Cem Kaner and James Bach argue for a creative approach to testing in which the tester is guided by intuition, not by a method or a tool. A good, creative tester will sense where the bugs are.
This approach amounts to exploring the software. The tester pushes into the system and follows paths that might lead to failures. If none turn up, the tester goes back and follows a different path. With years of experience, a seasoned tester knows where to look. Developers talk about a “code smell” when they see suboptimal code. In the same way, you could say testers pick up a “bug smell.”
We’ve made another interesting observation in our projects. Good, experienced testers who work exploratively use the structured test techniques implicitly: they have already internalized the basic ideas.
And that’s only where test techniques begin. Session-based testing, acceptance test-driven development and behavior-driven development build on the team’s shared responsibility for quality and help make quality part of the whole process.
So What about the Test Manager?
Finally, there’s the question of roles in the test process: does an agile project still need a test manager? The answer is very clear: it depends. In any professional project environment, traditional or agile, a lot of quality assurance tasks obviously have to be organized and carried out. In the traditional approach, a test manager is at the center of test planning and control. An agile team is supposed to organize quality itself. But especially in agile projects, a team can benefit from being able to turn to someone who has testing and quality assurance know-how, knows the test techniques and can coach the team in its testing activities.
Frequently Asked Questions
Is agility just a project management method?
No. Agile is often marketed that way because ideas packaged as toolkits, methods, and approaches sell well. A more sustainable understanding of agility is as a culture and mindset: a team pursues its goals in a self-organized, courageous, and transparent manner. This leads to greater employee self-efficacy, increased engagement, higher satisfaction, and a greater sense of flow when carrying out tasks.
How can you tell if a team has truly internalized agility?
One indicator is the results of retrospectives. The team constructively questions the process as it’s being practiced and adapts it to the specific situation. This leads to the development of a unique approach that fits the team and its context and gives individuals room to express their skills and interests. The true value only becomes apparent once values are internalized and processes are established.
What is the purpose of the test pyramid model in agile projects?
The test pyramid organizes the test types to be performed, including their frequency of use, costs, and dependencies between test types and test levels. Automated unit tests often form the foundation; they run alongside development and precede all other tests. Depending on the system architecture, various intermediate stages follow, culminating in manual tests as part of customer acceptance.
Why is the V-model ill-suited for test planning in agile teams?
In most agile teams, the integration levels are very closely aligned and are barely separated in terms of personnel. As a result, the clear assignment of who performs which review or test and when loses its basis. For some Kanban teams, it still works reasonably well because Kanban is flexible in this regard. Nevertheless, testing experts often rely on this approach, even in projects that do not follow the V-model.
What do the four testing quadrants accomplish?
They divide testing into four distinct groups based on two dimensions and help organize dynamic testing and static testing, as well as functional testing and non-functional testing, into a single framework. Janet Gregory and Lisa Crispin developed the model based on the question of what tests agile projects need and who should ideally perform them, building on ideas from Brian Marick.
Are traditional test techniques, such as equivalence partitions, still needed in an agile environment?
Their ideas remain relevant: grouping similar behaviors as in equivalence partitions, examining boundary cases as in boundary value analysis, and checking logical and technical states as in state-based testing. Agile projects usually lack the time for an explicit, step-by-step approach. That’s why internalizing the ideas matters more than following a procedural workflow. In safety-critical environments, however, this approach should be reevaluated.
Does exploratory testing replace structured testing?
No, it complements it and is gaining importance in an agile context. The tester is guided by intuition, not by a method or a tool: they delve into the system, follow potential failure paths, and turn back if they find nothing. Experienced exploratory testers implicitly use structured methods because they have internalized their underlying principles.
Who is responsible for test planning and quality in an agile team?
The team manages quality itself, whereas in the traditional approach, a test manager is at the center of test planning and test control. This does not mean that quality assurance tasks disappear. It is beneficial to have someone who brings test and quality assurance expertise to the table, is familiar with test techniques, and coaches the team on testing activities. Whether a separate role is necessary for this depends on the project context.
