Test design techniques are structured methods testers use to produce test coverage they can demonstrate. They fall into four groups: process-oriented, condition-oriented, data-oriented and appearance-oriented. Around ten techniques drawn from all four groups are enough for about 90 percent of testers throughout their careers.
Key Takeaways
- There are 25 to 30 test design techniques in total, but most testers only need about ten of them in their day-to-day work.
- All test design techniques fall into four groups: process-oriented, condition-oriented, data-oriented and appearance-oriented, and every tester should master at least one technique from each group.
- Knowing techniques from only one group is a trap: a condition-oriented technique won’t help when the actual problem lies in the data.
- Developers often test only the happy path in unit tests and neglect error cases, even though equivalence partitioning alone is enough to close that gap.
- Templates for techniques such as decision tables make them much easier to learn, because learners fill in a ready-made structure instead of building the table themselves.
Why Testers Often Ignore Test Design Techniques
Many testers know test design techniques but don’t use them. Rik Marselis has been teaching these techniques for almost 25 years, and this gap between knowing and doing is exactly what frustrates him.
The first reason is mundane: nobody requires testers to use a technique. Where nobody expects it, the structured approach is the first thing to go.
The second reason has to do with the quality of the system. If the software quality is poor, you don’t need a sophisticated technique. You press a button and you’ve found the bug. Only when quality is good does a technique show its value, because it proves what coverage was actually achieved.
The third reason is the effort of learning them properly. Passing a certification exam means you know the basics and can tick answer A, B, C or D. Real-world use is more complicated than any exam question, and that is where many people drop out.
With 25 to 30 Techniques, Quantity Becomes the Problem
Anyone who works through ISTQB Foundation, Advanced Test Analyst and Technical Test Analyst learns roughly 15 to 20 test design techniques. There are about 25 to 30 in total, depending on whether you count variants as separate techniques.
That many is overwhelming. Earlier TMAP certifications required knowledge of 19 techniques and approaches, which simply couldn’t be taught in a three-day course. The result was an exam people passed without having really learned anything.
TMAP drew its conclusions. The entry-level course teaches five techniques and one approach, with enough time and practice to actually use them at beginner level. Fewer techniques, but ones people can apply.
Four Groups of Test Design Techniques
All test design techniques can be sorted into four groups. This classification, developed by Rik and his colleague Bert, makes the field manageable.
- Process-oriented: path testing with different levels of path coverage, state transition testing.
- Condition-oriented: decision tables, where conditions determine the test cases.
- Data-oriented: equivalence partitioning, where a data element has several partitions.
- Appearance-oriented: syntactic checks of buttons, fonts and colors, plus non-functional aspects such as performance and usability.
The practical takeaway is clear: master at least one or two techniques per group. If you only know techniques from a single group, you walk into a trap.
One team Rik coached chose the elementary comparison test as its standard because it promises high efficiency and effectiveness. In practice, it didn’t work. The team’s problem was data-related, and a condition-oriented technique is the wrong choice for that.
Which Techniques Make Up the Basic Toolkit
The entry-level course covers all four groups with five techniques and one approach: equivalence partitioning and boundary value analysis from the data group, path testing from the process group, the decision table from the condition group and syntactic testing from the appearance group. Exploratory testing comes on top as an experience-based approach.
The second TMAP course, “High-performance quality engineering”, adds one more technique per group. Together, that makes about ten techniques and two approaches.
That toolkit covers most of the work. Around 90 percent of testers never need more than this over their entire career.
How to Choose the Right Test Design Technique
Four criteria decide which test design technique fits.
- The type of test problem: Which of the four groups do you pick from?
- The quality characteristic: If it’s about performance, for example, you’ll quickly end up in the appearance group.
- The risk: Higher risk calls for more coverage.
- The skills of the people involved: If the team doesn’t know a technique, you either pick one they know or train them.
Three risk levels are enough: low, medium, high. Percentages are misleading. If you demand 72 percent, you have to explain why not 69. Since the techniques usually offer only three coverage options anyway, three risk levels match them exactly.
Templates Make Techniques Easier to Learn
Templates make techniques learnable that used to be considered hard. One TMAP trainer admitted that for years she had found decision tables difficult to teach. A freely available template changed that.
The reason is simple. With a template, you no longer have to explain how the table is built. You show the finished structure and the learners fill it in: set the conditions, define the actions, and work out the expected result from true/false combinations. The expected result is the core of every test case.
Not every technique needs a template. Boundary value analysis is so intuitive that people often use it without ever having heard the term.
Path Testing Is the Underrated Technique
Path testing covers all paths through a business process and is especially well suited to acceptance testing. The only process-oriented technique ISTQB teaches is state transition testing, which requires states. But not many systems have states.
In acceptance testing, you want to check business processes, not states. That is exactly what path testing, also known as the process cycle test, is made for.
In a current project, business analysts are doing the functional acceptance testing. Because they had good process flows, Rik suggested path testing to them. After an hour and a half of explanation with exercises and a two-hour session on a real process flow, the first test cases were ready, built step by step from a template.
Coverage and Experience Go Together
A technique on its own isn’t enough. It proves that you’ve covered everything that matters for your risk level. Confidence, though, only comes from combining coverage with experience.
In that project, the analysts found a situation their test cases didn’t cover, even though every path had been covered. Rik’s advice: add a test case based on your experience. But don’t throw any away, because you need the covered cases as proof.
“This is where coverage-based testing meets practice-oriented testing. Add a test case from your experience, but don’t throw one away, because you need it for your coverage.”
(Rik Marselis)
The higher the risk, the more thorough the test planning. Experience-based techniques such as exploratory testing should always be combined with planned techniques.
Developers Benefit from Simple Techniques Too
In unit testing, developers often go by code coverage, but “100 percent coverage” says little as long as it’s unclear which kind is meant.
Line coverage is the weakest measure. Some developers pack five statements into one line, and a single executed statement already produces 100 percent coverage of that line. Statement coverage is better, because every statement has been exercised.
Decision coverage is the most valuable, because decisions are the most important statements in the code. A move statement will very likely work. With if statements and while loops, though, you want to know whether the logic handles both the good case and the error case.
This is where equivalence partitioning, the simplest data-oriented technique, already helps. Many developers only test the happy path and forget the error case. The technique gives you one test case per class, the valid one and the invalid one.
The real lever is awareness, not teaching. Tell developers they should also test what happens with wrong input, and it makes sense to them. Their reaction: then I’ll have twice as many test cases. Yes, you will.
The Techniques Stay, the Users Change
No new test design techniques are on the horizon. All the known ones appear in books that are more than 20 years old.
The earliest is “The Art of Software Testing” by Glenford Myers, around 45 years old, with boundary value analysis and equivalence partitioning. Then there is the work of Boris Beizer and the old TMAP books. ISTQB often cites Beizer and, in part, Myers.
What is changing is who uses the techniques. When business analysts apply path testing to their business processes, it matches exactly how they see the system. The technique is old; the user is new.
Three Techniques That Hold Up in Practice
For Rik, three techniques stand out.
Path testing comes first because it fully covers business processes and works in practice. The decision table reliably checks every single possibility, but gets out of hand with more than five conditions because it grows too large. In most cases, far fewer conditions are enough, so it works well.
The elementary comparison test is the most complex of the three and delivers the most thorough coverage. It tests multiple decision points, each with modified condition/decision coverage. From these test situations, test cases are combined from the start to the end of a process.
Its strength shows where path testing doesn’t cover enough. If different combinations of outcomes lead to the same paths, path testing yields only one test case, while the elementary comparison test yields all of them. That means a few more test cases, but the highest coverage you can achieve. If a system passes them, you can trust that it works.
Learning this technique properly takes a full-day workshop, though. The right template already exists.
Frequently Asked Questions
How many test design techniques does a tester actually need to master?
About ten techniques and two approaches are sufficient for the majority of the work, even though there are about 25 to 30 in total. Those who complete the ISTQB Foundation, Advanced Test Analyst, and Technical Test Analyst certifications learn 15 to 20 of them. What matters isn’t the quantity, but mastering at least one technique from each of the four groups. About 90 percent of testers won’t need more than that throughout their entire careers.
Is structured test design worthwhile even when software quality is poor?
Hardly. If the quality is poor, you can find defects just by trial and error: press a button, find a defect. Techniques only demonstrate their value when quality is good, because that’s when they prove what coverage was actually achieved. A second reason for their limited use is just as simple: Often, no one requires testers to use any technique at all.
Should a team commit to a single standard test design technique?
No. One team chose the elementary comparison test as its standard because it promises high efficiency and effectiveness. In practice, this didn’t work: The problem was data-related, and a condition-based technique is the wrong choice for that. It makes more sense to master one or two techniques from each of the four groups.
How finely should the risk be graded when determining test coverage?
Three levels are sufficient: low, medium, high. Percentages are misleading, because if you require 72 percent, you have to explain why 69 percent isn’t enough. The techniques themselves usually offer only three coverage options anyway, so three risk levels fit perfectly. The higher the risk, the more thorough the test planning will be.
Which technique is suitable for functional acceptance testing of business processes?
Path testing, also known as process cycle testing, covers all paths of a business process. The only process-oriented technique ISTQB teaches is state transition testing, which requires states; however, not many systems have states. In one project, business analysts created their first test cases after an hour and a half of explanation with exercises and a two-hour session using a real business process.
What should you do if experience suggests a test case that the technique doesn’t provide?
Add it, but don’t discard any of the existing test cases. You’ll need the cases derived from the technique to demonstrate coverage. In one project, analysts encountered a situation that their test cases did not cover, even though all paths had been addressed. Trust arises only from a combination of coverage and experience.
What does 100 percent code coverage in unit testing mean?
Not much, as long as it remains unclear what kind of coverage is being referred to. Line coverage is the weakest metric: if a developer packs five statements into a single line, a single executed statement already counts as 100 percent coverage for that line. Statement coverage is better. The most valuable is decision coverage, because with if statements and while loops, it shows whether the logic handles both the normal case and the error case.
When does a decision table reach its limits?
With more than five conditions, it gets out of hand because the table becomes too large. With fewer than that, it has high reliability and checks every single possibility, and in most cases, you can get by with significantly fewer conditions. If you need more coverage, use the elementary comparison test, which checks multiple decision points but requires a full-day workshop to learn.
