Model-based testing refers to an approach in which test cases are systematically derived from a formal model of system behavior. The central problem in practice is the gap between the abstract model and concrete test execution. A two-phase approach bridges this gap: First, low-level test cases are created from the model, then concrete test data and automation scripts are added interactively.
Key Takeaways
- Model-based testing often fails in practice due to the gap between abstract test design and concrete test implementation, because concrete test data and GUI actions make the model unnecessarily complex.
- A two-phase approach solves this problem: first, high-level test cases are generated from the model and kept manually executable; only then are concrete data and automation scripts added.
- According to Capgemini’s World Quality Report, test automation has not yet achieved the set business goals: neither smooth CI/CD pipelines nor efficiency increases nor measurably better software quality.
- Systematic test design procedures such as equivalence partitioning or boundary value analysis are hardly used by trained testers in practice; instead, gut feeling dominates when writing test cases.
What model-based testing is and where it gets stuck
Model-based testing derives test cases from a model instead of writing them individually by hand. The tester defines what is to be covered, models it and generates the test cases from it. The idea is old and attractive. In practice, it rarely runs smoothly.
Matthias Hamburg has introduced the approach as a test manager in several projects. His verdict is sober: “It never went so smoothly.” This is in line with industry observations. The World Quality Report by Capgemini and Sogeti states that, overall, test automation has not achieved the business goals associated with it.
Three expectations remained unfulfilled: smooth CI/CD operations with automation, the hoped-for increase in efficiency and measurably better software quality. Managers and company management were interviewed who were looking at business objectives. Exactly these goals were missed.
Why most MBT approaches fail in reality
Two hurdles stand in the way of model-based testing. The first concerns the people, the second the tool workflow.
The first hurdle: Those who create models are usually not trained in modeling. Formal models require training that testers rarely have. Therefore, only models that remain understandable without a long lead time and are close to the tester’s way of thinking are practical.
The second hurdle lies in the process itself. Model-based testing does not currently offer a seamless transition from modeling to the actual test. It is precisely in this gap that the problems arise.
Where the gap is in the test process
The ISTQB model of test activities clearly organizes the steps: test analysis, test design, test implementation, test execution. The test design defines what is to be covered, i.e. the test conditions. Test cases can then be generated from the model. Execution can be automated using tools such as Selenium or Playwright.
In between lies test implementation, and this is the crux of the matter. This is where concrete test data is selected, where it is determined which buttons are pressed and which concrete results are checked. It is the step from the abstract to the concrete.
The usual response of MBT tools is to pack the concrete data into the model. However, this makes the model very complex. A model should abstract, provide a logical view. Including concrete data breaks this view. This overloading is precisely one reason why model-based testing has not become established.
The two-phase approach closes the gap without overloading the model
A more recent approach separates abstraction and concretization into two phases. First, high-level test cases are created from the model. These must be manually executable, because in manual testing, people fill in the gaps themselves anyway.
Phase one can be completed early, before the software is finished. This is the shift left advantage: modeling early ensures the quality of the test basis and the specification. Then you stop without fear of a paradigm shift.
Phase two begins as soon as the real application is running. The tool executes the high-level test cases and asks back at the open points: What does “enter text” mean in concrete terms here? It displays the dialog interface, you enter the appropriate text and have the action recorded.
This is not a simple capture-and-replay. Earlier capture tools generated scripts that often no longer worked afterwards, with corresponding maintainability problems. Here, the action is first generated at GUI level from the functional level. This action is readable and can be re-edited in a low-level test case. The executable script for the automation platform is then created from this, in the tested case Playwright.
Why this approach remains maintainable
Maintainability is the real benefit. If an action changes because the implementation has changed, all places where this action occurs are updated. The redundancy is not formally reduced, but the tool keeps it consistent.
A typical practical problem demonstrates the benefits particularly clearly. Some UI frameworks generate new IDs for the GUI elements with every small change. As soon as a button is added somewhere, everything behind it is renamed. This breaks classic automation.
The tool handles this pragmatically. It stops at the point where it cannot find a selector and asks for it. You adjust the selector in the list and then the test runs again. Only what has actually changed needs to be updated.
Action State Testing: a state model in the language of testers
The tried and tested tool comes with its own model, coupled to a black box procedure called Action State Testing. In terms of content, it is close to state-based testing, but is formulated closer to the way testers work.
The model is currently text-based, not graphical. You write down test steps in the same way as you are used to when testing: an action, an expected result to be tested, a subsequent state. The subsequent state of a step is the prerequisite for the next one.
An example from the glossary application makes this clear. Initial state: the search is displayed. Action: enter a search text. Check: the number of hits is at least one and the search text is included in the hits. Subsequent state: the results list is displayed. If you select an element, the details appear and a back button takes you back to the hit list.
The sequence of steps can be displayed as a flow chart. The result is scenario-based and also uses the states, a test-relevant state model that can be systematically covered.
One procedure alone is not enough
Model-based testing does not replace the classic design methods, it combines them. In addition to the state-based approach, equivalence partitioning was also used in the glossary app. We searched not only for texts from the name of a term, but also for abbreviations and synonyms, because the search had to work for all three.
This is a common weakness in practice. Many testers learn equivalence partitions, boundary value analysis, state and decision tables once, but tend to apply them on a gut feeling basis in everyday life. A structured approach, combined with modeling, brings these procedures back into daily work.
A negative example from industry shows just how far things can go without such discipline. A large company with quarterly releases had all the specialists come into the office on the release weekend and test on an ad hoc basis. When asked when any bugs found would be fixed, the answer was: “We hope there aren’t any.” A clear sign that trained testers were not doing their job and did not know any test techniques.
Open construction sites of the approach
No tool is ready, and this approach is new to the market. Two wishes remain.
- Data-driven testing: Test data such as the search text should be stored separately and reused in further test cases instead of just being a loose text reference in the step.
- More design procedures: In addition to state-based testing, equivalence partitioning and boundary value analysis directly in the tool would be helpful in order to achieve the desired test coverage cleanly.
Despite these gaps, the application could be tested extensively. Even if the glossary app is not commercially critical and nobody loses a cent, it remains a question of attitude: a testing expert will not hand over an application that has not been well tested.
How to get started with model-based testing
People often shy away from model-based testing because it is unclear what they can actually do with it. The pragmatic way to get started is through trial and error, not theory.
Download a trial installation of a good tool, there are several. Try out how it works and see what suits you and what you can handle easily.
If I have to study for days before I can even generate the first small test case, then don’t bother.
- Matthias Hamburg
Accessibility is the benchmark. One advantage of the tried and tested tool is the no-code generation. As a specialist tester, you specify directly in the GUI what is to be tested and how it is to be entered, without having to write an automation script. In previous large-scale projects, this required a separate sub-project consisting of automation specialists who implemented what the specialist testers had designed.
Frequently Asked Questions
Does test automation meet the business goals that companies associate with it?
No. The World Quality Report by Capgemini and Sogeti finds that test automation has so far failed to meet the set business goals. Three expectations remain unmet: seamless CI/CD operations with automation, the hoped-for increase in efficiency, and a measurably higher software quality. The survey included managers and senior executives—in other words, people who focus specifically on these business goals.
What qualifications must testers have to work with models?
Those who create models in the testing team usually have no formal training in modeling. Formal models require training that testers rarely undergo. Therefore, the only practical models are those that remain understandable without a long learning curve and align closely with the tester’s way of thinking. One example is Action State Testing: action, expected result, subsequent state—noted in text form just like familiar test steps.
Why do models often become too complex in model-based testing?
Because many tools automatically include the specific test data in the model. A model is meant to abstract and provide a logical view. If you include specific data and GUI actions, this view breaks down and the model becomes unnecessarily complex. This overloading is considered one of the reasons why model-based testing has not yet gained widespread acceptance.
Is the two-phase approach just a new form of capture-and-replay?
No. Earlier capture tools generated scripts that often no longer ran afterward, leading to corresponding maintainability issues. With the two-phase approach, an action at the GUI level is first generated from the business logic level. This action is readable and can be edited later for a low-level test case. Only then is the executable script generated for the automation platform—in this case, Playwright.
What happens if a UI framework assigns new IDs to GUI elements with every change?
Traditional automation fails in this scenario: If a button is added somewhere, everything associated with it is renamed. A tool can handle this pragmatically by stopping at the point where it cannot find a selector and querying the user. The selector is adjusted in the list, after which the test continues. Only what has actually changed is updated.
Does model-based testing replace traditional test design methods?
No, it combines them. In the glossary application under test, equivalence partitioning was used alongside the state-based approach because the search for term names, abbreviations, and synonyms must function correctly. Many testers learn about equivalence partitions, boundary value analysis, state tables, and decision tables at some point, but tend to apply them based on gut feeling in their day-to-day work. A structured approach brings them back to a more methodical approach.
How can you tell if an MBT tool is suitable for beginners?
By its accessibility. Matthias Hamburg puts it this way: If you have to spend days learning before you can generate your first small test case, you should steer clear of it. It makes sense to download a trial version and try out what fits your own workflow. A plus is no-code generation, where subject-matter experts define what’s being tested directly in the GUI.
