A lightweight test plan is a modular kit of short, linked topic booklets that are used specifically for individual situations rather than as a one-off overall document. Each module contains a user story, a description of the topic, a self-assessment to measure maturity and concrete improvement steps.
Key Takeaways
- The Federal Office of Administration’s QA toolkit replaces heavyweight test manuals, some of which are 300 pages long, with modular, short building blocks that employees can use independently depending on the situation.
- Building traceability between test cases and requirements from the outset costs little effort, but introducing it retrospectively is a huge amount of work.
- A monthly Community of Practice with regularly 70 to 80 participants has resulted in more knowledge transfer at the Federal Office of Administration than training courses, because project teams exchange concrete solutions directly with each other.
- External testing by service providers has led to the loss of internal testing know-how, which makes the toolbox necessary as a tool for regaining this knowledge.
- The toolkit is to be published in Q1 under a Creative Commons license on an open code platform so that other authorities can also benefit from it and contribute their own adaptations.
Why heavyweight test plans fail in administration
Hundreds of pages of test manuals end up in a drawer and are never read. This is exactly the pattern that authorities experience again and again with classic test plans and test strategies. The documents are created because a process demands them, not because someone needs them.
The Federal Office of Administration is traditionally committed to the federal government’s V-model XT. This works well as long as the requirements are clear, for example if a law or regulation specifies a certain specialist procedure. It becomes difficult as soon as the situation changes quickly.
The so-called refugee crisis in 2015 was the trigger for Oliver Kortendick and his team to rethink their approach. A procedure had to be in place in a short space of time, even though the law had not yet been passed. Development began and the procedure was continuously adapted to the legal situation. This would hardly have been possible with a rigid approach.
A large organization is the natural anti-pattern for agility
The Federal Office of Administration embodies almost every condition that makes agile working difficult. Over 6000 employees, spread across 20 properties in Germany, numerous external service providers, a differentiated hierarchy and specialist departments that are separate from IT. This creates sluggishness and slowness.
Added to this is the sheer diversity of the work: over 150 specialist procedures, from the smallest to the largest projects. Everyone is calling for standardization, but standardization is difficult to achieve with this range.
In this environment, agile transformation only works as a path of small steps. Simone Mester and Oliver do not describe a big bang or a purchased framework, but rather a slow transformation. It is not complete even after years, but it is moving.
Testing expertise must not only lie with external parties
If external service providers take on the majority of testing work over the years, the in-house organization loses its knowledge. This was precisely a central problem at the Federal Office of Administration. The question became: How does the testing knowledge get back to our own colleagues?
The starting point was a community of practice that has been in existence for over five years. It meets six to ten times a year, regularly with 70 to 80 participants. That is a high figure for a voluntary format.
The focus is on workshop reports. People from the projects talk about what they have done and contacts are made. Someone listens and realizes that another team has been working on exactly the same problem for months. This pragmatic transfer of knowledge replaces the need to reinvent everything in every project.
Textbook solutions are of little help because they often do not fit the authority. What is needed are approaches that have worked once and can be adopted elsewhere.
What the QA toolbox is
The QA toolkit is a modular knowledge model from which you can select exactly the module that solves your current problem. Instead of reading through a test manual of 120, 150 or 300 pages, you select a single module, comparable to a booklet from a bookcase.
The content of the kit is based on the ISTQB process model, the ISO software quality characteristics and the team’s own experience. The whole thing is set up in Confluence and the pages are linked to each other.
The topics cover the practical aspects that tend to get lost in public authority operations:
- Requirements management
- Test levels such as component, integration and system testing
- Test design procedures
- Traceability of test cases to requirements
- Types of review
- Accessibility
- Test data
Traceability is a good example of the benefits. If you do it right from the start, the effort is minimal. If you try to pull it in afterwards, from the test cases back to the requirements, it becomes a huge amount of work.
How a building block is structured
Each module follows the same dramaturgy and therefore appeals to different types of learners. The ISTQB definition is at the top so that everyone involved has a common wording. Especially with changing service providers, people otherwise talk past each other phenomenally.
This is followed by a user story. These stories are made up, but could have happened in the office. Reading them is an easy introduction to the topic and makes you smile. The hope behind it: If you smile once, you will read the rest.
This is followed by the simplest possible description of why the topic exists in the first place. Some content is also conveyed via short YouTube videos, because many people find two minutes of video more enjoyable than a long text.
There is a maturity model for each module that is tailored to this specific topic. It ranges from “initial”, where something only happens ad hoc, to controlled stages. This allows you to categorize yourself and see where you stand.
At the end there are in-depth links, from book tips to internet sources and scientific articles. A small knowledge database collects topics that are too small for a separate module. There are also templates that are in high demand, for example for protocols or for the question of how to write a test case in the first place.
Reasoning contexts are more important than regulations
People only improve if they know why. This realization came from many feedback rounds with our own employees and shapes the toolkit. People want to understand the context, not just follow another instruction.
The knowledge model is therefore multi-layered. You can stay on the surface and get a quick overview. Or you can go into depth via the hypertext, right into scientific articles.
There is a cultural issue behind this. In the classic silo, everyone sat on their own, threw their work results over the fence and no longer saw the overall process. Agile demands the opposite.
In agile, you want the team to share responsibility and commit to the highest product quality. This means that everyone has to get out of the silo and contribute their knowledge and energy. Oliver Kortendick
How the toolbox works in the project
The toolkit has been in use in the field since July. A project in a difficult situation was deliberately chosen for this: with communication problems where people sometimes stopped talking to each other.
This project did not have a risk-based test strategy. There were no priorities, testing was scattered evenly across everything. The result was considerable delays.
Three areas were identified from the toolbox in which the project could improve by the end of November. Most of these points had already been worked through. At the same time, the team regularly and simply measured the mood: how are you feeling right now? These values improved, and this is precisely what is relevant for team building and good collaboration.
Exercises close the gap between reading and ability
Some people only learn when they try something out for themselves. That’s why there will be a four-hour exercise for each module. Three are ready, more will follow next year.
An exercise can be very concrete, such as writing a test case with boundary value analysis on paper while the facilitators sit next to it and the participants coordinate with each other. You can explain a procedure to someone a hundred times, some people just have to do it once.
The exercises are organized according to roles. BVA works with familiar roles such as component manager or project manager. You don’t have to do all the exercises, but five or six that fit your role and help you progress.
You have to ask for reports, not just receive them
Reporting doesn’t just mean writing reports yourself, but receiving, understanding and actively requesting reports. This is a focus for the operational level.
The counter-example is widespread: External service providers deliver a convolute of HTML pages according to the motto “a lot helps a lot”. This leaves you on your own and only gets you a little further. The better way is to clearly state what information you need and in what form.
Why the Federal Office of Administration publishes the construction kit
The QA toolkit will be released in the first quarter under a Creative Commons license, most likely on the Open Code platform. You will then be able to view it, download it and modify it to suit your needs.
The logic behind it is simple. A lot of work and money has gone into the toolkit, other authorities have the same problems and there are already a number of requests. What works should also benefit others.
It would be nice if changes flowed back because the team itself wants to continue learning. It cannot accept any liability or warranty. And sometimes the benefit lies in learning from someone else’s mistakes.
