Model-based testing describes an approach in which processes are graphically mapped as a model in order to develop requirements together with the specialist department and testers and automatically generate test cases from them. The model replaces continuous text with a visual language, makes gaps in requirements immediately visible and also serves as a basis for impact analyses in the event of changes.
Key Takeaways
- Model-based testing creates a common visual language between the business department, testers and developers that steers discussions away from detailed technical debates and towards relevant issues.
- Around half a year’s work at Lufthansa CityLine resulted in around 700 test cases, which are stored in a structured manner in 60 models and generated at the touch of a button.
- The model makes gaps in requirements immediately visible: a kick-off workshop simultaneously delivered improved requirements and the appropriate test cases.
- Changes in operations, such as new wage agreements, can be carried out as a targeted impact analysis using the model without having to sift through hundreds of test cases individually.
- If you design models that are too complex, you risk a test case explosion; small, understandable models with a narrow scope have been proven to work better than trying to map everything in one.
Model-based testing brings stakeholders to the table
Model-based testing solves a basic problem in large software projects: The knowledge needed for good testing is distributed among three parties. Technical experts know the processes. Testers know how good test cases are created. Developers build the software based on the information. Bringing these three together is the real work, and a model creates a common basis for this.
A model works like a language. It is a way of expressing something that everyone involved can agree on. Because it is presented graphically, it remains simple and understandable. Marvin Jakob describes the core effect as follows: “You can point your finger at something and say, we need to talk about this again. It is precisely this visibility that is missing in pure continuous text.
At Lufthansa CityLine, the approach was used in a crew management system that was set up as part of a newly founded airline. The challenge with an externally commissioned system is always the same: does the supplier deliver exactly what was ordered? In order to be able to accept this, you need specialist knowledge and communication between the specialist department and the test.
How is a test model created from distributed knowledge?
A test model is created using interview techniques, not from ready-made diagrams. Such models do not lie around in a project, they are developed in a question-and-answer game.
The starting point is a single requirement and the question of how it would be tested. What is the first step, what is the second? These steps end up as activities in the model. From there, the model leads almost automatically to the next question: What happens next? Does anything else happen here? Do we need a branch at this point?
The procedure is tangible and flexible. Activities can be moved, gaps can be inserted, forgotten steps can be added. At some point, the model reaches an end point that has not yet been reached and it becomes clear which activity is still missing. At the end of a workshop, there is a tangible result, which is not a given in software development.
Start at a high altitude, not in detail
The most important practical rule in model-based testing is: start at a very high altitude and get through from start to finish before you talk about details.
If you get bogged down in sub-diagrams and low-level branches right at the start, you will lose the overview. The opposite approach is better: build a high-level model, talk about it and only when everyone has understood it, go one level deeper and introduce more test information. Step by step, until ready-made test cases can be generated at the push of a button.
This approach has a tangible advantage in the discussion. Discussions move away from technical details and minutiae to the relevant issues. If you stay at a high level, you stay in solution mode instead of getting lost in the perceived infeasibility.
The sorting of what is tested and what is not shifts to the back. You first bring the technicality into the model and decide on the coverage later: edge coverage, full-path coverage, test data variants, equivalence partitions. This decision can be made in a relaxed manner without the model suffering as a result.
Good testing requires good requirements
You can only write good tests if the requirements are good. Requirements and tests form a unit because together they describe the feature.
The model makes gaps in the requirements obvious. It is possible to discuss very specifically where improvements need to be made. At CityLine, the team came out of the kick-off workshop with improved requirements and the appropriate tests, both in one step.
The workshop had a clear sequence: in the morning, a refinement of the user stories brought along was carried out in order to understand them, with initial improvements. Then someone who had never used the tool before sat down with it and built the first test model. At around three in the afternoon, the participants went home with refined stories and the test cases generated for them.
Keep models simple, otherwise they won’t work
A good model is simple and understandable, not an attempt to map the whole world with sub-diagrams. Arbitrarily complex models rarely work.
A practical test for comprehensibility: get someone else to join you who has not seen the model before and have them explain what it says. If this works, the model is formulated simply enough in the language of the model. Aviation processes are complex, but this can be implemented for numerous cases.
A clear test scope is just as important. There is no single model that maps all business processes. If you pack everything into one model, you produce a test case explosion and, in the worst case, the computer comes to a standstill during generation. Focusing on one scope protects against this.
From model to test case at the push of a button
Once the model is in place, the test cases can be generated with the Generate button. The hard work of preparing tests so that they can be executed is done by the tool, which means that part of the process is automated.
At CityLine, around 700 test cases were created in this way, which are distributed in around 60 models. The models could be transferred to Jira, where they could be viewed, annotated and reviewed. The existing infrastructure was used instead of building a new one.
In Jira, the model is also visible in color: what was tested, what works, what doesn’t. This creates a basis for communication with the stakeholders and builds trust because the work remains transparent and can be explained at any time.
I can communicate very well with my stakeholders about this, build trust and work with total transparency. That’s what’s important to me in testing.
Oliver Schuhmacher
This keeps the test basis up to date in the event of changes
The model simplifies the impact analysis when requirements change. Instead of going through pages of Jira lists, you look at the model and see the affected test cases behind it.
In the aviation environment, new collective agreements and rules are regularly added that need to be taken into account. With 700 test cases, it is a pain to check each one individually for validity. If the models are stored in a structured way, you can quickly see which areas are affected and where the change needs to be entered.
The test cases can be overwritten or versioned directly from the adapted model. This keeps them up to date. There is still work involved, but the effort goes into the relevant questions: What has changed, what is the best way to test it?
The model should live on, not end in the project
A test model belongs in the analysis phase, on a par with the requirements. It is at the beginning, when the team understands what it wants to build and where it wants to go.
In order for the benefits to be fully realized, the model should continue to exist beyond the end of the project. At CityLine, the models should be handed over to the line organization as a kind of documentation during the transition phase. If you throw the model away at the end of the project, you only gain half the benefit.
The prerequisite for this is a team of testers, specialist departments and developers who work together over the entire period through to operation. In large projects, not all requirements are available from the outset, which is almost impossible. The model is the basis on which this team continuously exchanges and documents information.
Just get started and try it out
The most important advice for getting started is to try it out. There is a learning curve with a new tool and you will inevitably make mistakes at the beginning.
Nevertheless, the approach saves time in the end. Because a lot of effort goes into the analysis and the right things are discussed there, you are largely finished afterwards. There is little you can do wrong, even if you approach it bit by bit.
If you are unsure, you don’t have to start alone. The testing community is a good place to go for help, coaching or a second look at the model. In essence, the value lies where distributed knowledge comes together, requirements are sharpened and the tests that count for business value emerge directly from the model.
