Skip to main content

Search...

Podcast Software Testing

Test design methods

25 to 30 test design techniques exist, but most testers hardly use any of them. Why five of them are enough for 90% of all testing tasks.

• • 9 min read
Cover for Test design methods

Test design techniques are structured methods that testers use to generate verifiable test coverage. They can be divided into four groups: process-oriented, condition-oriented, data-oriented and appearance-oriented. Anyone who masters at least one technique from each group covers around 90 percent of typical test situations in practice.

Key Takeaways

  • There are between 25 and 30 test design techniques in total, but most testers only really need about ten of them in their day-to-day work.
  • All test design techniques can be divided into four groups: process-oriented, state-oriented, data-oriented and appearance-oriented, and a tester should master at least one technique from each group.
  • If you only know techniques from one group, you are falling into a trap: a state-oriented technique will not help if the actual problem is data-based.
  • Developers often only test the happy path during unit testing and neglect error cases, although equivalence partitioning is enough to close this gap.
  • Templates for techniques such as decision tables significantly lower the barrier to entry because learners no longer have to construct the table structure themselves, but fill it in directly.
Listen on Apple Podcasts Listen on Spotify Listen on YouTube

Why testers often ignore test design techniques

Many testers know test design techniques but don’t apply them. Rik Marselis has been teaching these techniques for almost 25 years, and it is precisely this gap between knowledge and practice that frustrates him.

The first reason is banal: No one requires testers to use a technique. Where there is no expectation, the structured approach is the first thing to fall away.

The second reason depends on the quality of the system. If the software quality is poor, there is no need for sophisticated technology. You press a button and you have found the error. Only with good quality does a technique show its value, because it proves what coverage has actually been achieved.

The third reason is the hurdle of proper learning. Anyone who passes a certification exam knows the basics and can tick answer A, B, C or D. The real application is more complicated than any exam question, and this is where many drop out.

Too many techniques are themselves part of the problem

Those who go through ISTQB Foundation, Advanced Test Analyst and Technical Test Analyst learn around 15 to 20 test design techniques. There are around 25 to 30 in total, depending on whether you count variants as a separate technique.

This quantity is overwhelming. Previous TMAP certifications required knowledge of 19 techniques and approaches, which simply could not be taught in a three-day course. The result was an exam that you passed without really having learned anything.

TMAP has drawn a conclusion from this. The entry-level course teaches five techniques and one approach, with enough time and practice to actually use them at beginner level. Fewer techniques, but applicable.

Four groups organize the entire field

All test design techniques can be categorized into four groups. This categorization, developed by Rik and his colleague Bert, makes the field manageable.

  • Process-oriented: Path testing with different levels of path coverage, state transition testing.
  • State-oriented (condition-oriented): Decision tables where conditions define the test cases.
  • Data-oriented: Equivalence partitioning where a data element has multiple partitions.
  • **Appearance: syntactic checks such as buttons, fonts, colors and non-functional aspects such as performance and usability.

The practical consequence is clear: master at least one or two techniques per group. If you only know techniques from a single group, you are falling into a trap.

One team that Rik accompanied as a coach opted for the Simple Comparison Test as the standard because it promised high efficiency and effectiveness. In practice, it did not work. The team had a data-related problem, and a condition-based technique is the wrong choice for this.

Which techniques form the basic set

The starter course covers all four groups with five techniques and one approach. Specifically, these are equivalence partitioning and boundary value analysis from the data group, path testing from the process group, the decision table from the condition group and syntactic testing from the appearance group. In addition, there is exploratory testing as an experience-based approach.

A second course on high-performance quality supplements one further technique per group. Together, this results in around ten techniques and two approaches.

This toolbox is sufficient for the majority of the work. Around 90 percent of testers need no more than this over their entire career.

How to choose the right technique

Four criteria decide which test design technique is right for you.

  1. The type of test problem: Which of the four groups do you choose from?
  2. The quality characteristic: If it’s about performance, for example, you’ll quickly end up in the Appearance group.
  3. The risk: High risk requires more coverage.
  4. The skills of those involved: If the team doesn’t know a technique, you either choose a known one or you train people.

Three levels of risk are sufficient: low, medium, high. Percentages are misleading. If you claim 72 percent, you have to explain why not 69. Since the techniques usually only offer three coverage options anyway, three risk levels fit in perfectly.

Templates lower the barrier to entry

Templates make techniques learnable that were previously considered difficult. One TMAP trainer admitted that decision tables were difficult to teach for years. A freely available template changed that.

The reason is simple. With a template, you no longer have to explain how the table is created. You show the finished structure and the learners fill it in: Set conditions, define actions, determine the expected result via true-false combinations. The expected result is the core of every test case.

Not every technique needs a template. Marginal value analysis is so intuitive that people often use it without ever having heard the term.

Path test is the underestimated technique

Path testing covers all paths in a business process and is particularly suitable for acceptance testing. ISTQB is the only process-oriented method that teaches state transition testing, which requires states. However, not many systems have states.

In acceptance testing, you want to test business processes, not states. This is exactly what the path test, also known as the process cycle test, is made for.

In a current project, business analysts are carrying out the functional acceptance testing. Because they had good process flows, Rik suggested the path test to them. After an hour and a half of explanation with practice and a two-hour session on a real process flow, the first test cases were ready, created step by step using a template.

Coverage and experience go together

A technique alone is not enough. It proves that you have covered everything that is important for your risk level. However, confidence only comes from the combination of cover and experience.

In that project, the analysts found a situation that their test cases did not cover, even though all paths were covered. Rik’s advice: Add a test case based on your experience. But don’t throw one away, because you need the covered cases for the proof.

This is where coverage-based testing meets practice-oriented testing. Add a test case based on your experience, but don’t throw one away because you need it for your coverage.

Rik Marselis

The higher the risk, the more thorough the test planning. Experience-based techniques such as exploratory testing should always be combined with planned techniques.

Developers also benefit from simple techniques

When it comes to unit testing, code coverage often counts for developers, but “100 percent coverage” says little as long as it remains unclear what kind is meant.

Line coverage is the weakest statement. Some developers pack five statements into one line, and one executed statement already generates 100 percent coverage of this line. Instruction coverage is better, so that every instruction has been checked.

The most valuable is decision coverage, because decisions are the most important instructions in the code. A move statement has a high probability of working. With if statements and while loops, however, you want to know whether the logic meets the good case and the error case.

This is where equivalence partitioning, the simplest data-oriented technique, helps. Many developers only test the happy path and forget the error case. This technique provides a test case for each class, the right one and the wrong one.

The real lever is awareness, not teaching. If you tell developers that they should also test what happens if they enter the wrong thing, it makes sense. The reaction: Then I have twice as many test cases. That’s right, you do.

The technology stays, the application moves into new hands

There are no new test design techniques in sight. All known techniques are in books that are over 20 years old.

The earliest is “The Art of Software Testing” by Glenford Myers, around 45 years old, with boundary value analysis and equivalence partitioning. There is also the work by Boris Beizer and the old TMAP books. ISTQB often refers to Beizer and partly to Myers.

What changes is not the canon, but who uses the techniques. When business analysts apply the path test to their business processes, this corresponds exactly to their view of the system. The technology is old, the user is new.

Three techniques that work in practice

From Rik’s point of view, three techniques stand out.

The Path test is at the top because it covers business processes completely and works in practice. The decision table checks every single possibility reliably, but gets out of hand if there are more than five conditions because it gets too big. In most cases, far fewer conditions are sufficient, so it works well.

The Simple Comparison Test is the most complex of the three and provides the most comprehensive coverage. It tests multiple decision points, each with modified condition-decision coverage. From these test situations, test cases are combined from the beginning to the end of a process.

Its advantage is evident where the path test does not cover enough. If different combinations of outputs lead to the same paths, the path test only provides one test case, while the simple comparison test provides all possibilities. This results in a few more test cases, but the highest achievable coverage. If a system passes these test cases, you can be confident that it works.

However, a full-day workshop is required to learn this technique properly. The right template exists.

Frequently Asked Questions

In order to successfully integrate test design procedures into agile development processes, they should be planned early on in the development cycle. Teams need to create tests together to clarify requirements and acceptance criteria. Automation of tests promotes quick feedback and adjustments. Regular reviews and retrospectives help to continuously optimize the test design process. Close interaction between developers and testers ensures that quality is taken into account from the outset.

Common challenges when using test design techniques are unclear test objectives and insufficient requirements, which can lead to ineffective testing. In addition, there is often a lack of resources, which makes it difficult to carry out extensive tests. Selecting the appropriate test design method is also critical, as not all methods are suitable for every project. Finally, the integration of test procedures into existing development processes and the training of employees can present additional difficulties.

To select the right test design method, first analyze the project requirements and objectives. Consider factors such as complexity, risk levels, test scope and available resources. For simple projects, methods such as equivalence class formation or boundary value analysis are sufficient. Complex projects may require structured methods such as test case-based approaches or exploratory testing. Also consider feedback from stakeholders and the available expertise in the team to maximize the efficiency and effectiveness of testing.

Specification-based test design methods offer the advantage that they are based directly on the requirements and specifications of a system. This ensures that all functional and non-functional aspects are tested. They enable the systematic identification of test cases, reduce the risk of overlooking important scenarios and promote the traceability of tests. In addition, errors can be detected and rectified at an early stage, which improves the quality of the end product.

The test design techniques for model-based testing include techniques for generating test cases from models that represent system behavior. The most common techniques include state diagrams, decision tree testing and activity diagrams. These techniques make it possible to systematically cover different scenarios and facilitate the identification of test cases based on the specified requirements. The use of these test design techniques increases the efficiency and effectiveness of testing by supporting automated test generation and early defect detection.

Risk-based testing as part of the test design process focuses on the identification and prioritization of risks. As a result, tests are targeted at the most problematic areas. Important principles are Understanding the application area and its risks, prioritizing test cases according to risk level and continuous risk review during the testing process. The aim is to use resources efficiently and minimize the probability of serious errors. This improves the quality of the software in a targeted and measurable way.

Experience-based test design methods offer the advantage that they are based on practical experience and real usage scenarios, which means that they often identify realistic sources of error. They should be used when comprehensive specifications are not available or the software is already in use. These techniques promote effective testing by integrating knowledge from previous testing and user feedback, which improves the quality of testing and increases testing efficiency.

Black-box test procedures check the functionality of a system without knowledge of the code, focusing on inputs and outputs. In contrast, white-box testing analyzes the internal code and structure to identify sources of error. While black-box tests simulate user perspectives, white-box tests aim to refine the program technology. Both test design methods are important, but offer different perspectives on the quality of the software.

There are various test design procedures that differ in their methodology. The main types include equivalent classes, boundary value analysis, state transition testing and exploratory testing. Equivalent classes group similar inputs, while limit analysis examines the extreme values. State transition tests visualize systems as state diagrams, and exploratory testing encourages creative approaches. Each method has its own strengths and is suitable for specific test objectives. Choosing the right test design method depends on the specific requirements of the project.

Test design techniques are systematic approaches to creating test cases to effectively find software defects. There are various methods, including equivalent partitioning, in which input values are divided into classes, and limit value analysis, which focuses on edge cases. Other methods include the decision tree technique and failure mode and effects analysis (FMEA). These methods help to maximize test coverage and increase the efficiency of the tests.

Share this page

LinkedIn X E-Mail
Back to all posts

Related Posts