When projects get into stormy waters, the quality of test reporting determines whether you are steering blindly through the fog or know exactly where you stand. A 3x3 matrix of test preparation, test execution and defect management is all you need to monitor test progress - provided you automate the database and measure its quality at the same time. The test management dashboard thus becomes a real management tool that not only shows you where you stand, but also predicts when you will really get there.
In this episode, I talk to Matthias Gross about a topic that many find dry, but is vital in projects: test reporting. Matthias shows his 3x3 matrix method, which he uses to monitor test progress - from preparation to execution to defect management. He not only automates his test management dashboard, but also monitors the data quality of his metrics with an internal control system. Instead of 1000 key figures, his approach delivers exactly the nine fields that really count - and he explains why good test reporting works like navigation on the high seas.
"Test reporting is a management tool." - Matthias Gross
Matthias Groß is a partner at TestGilde GmbH and has been working as a consultant for software quality assurance and test management since 2007. His focus is on operational test management, the introduction and further development of test management structures and the support of customer-specific test services. He is also involved in the Baden-Württemberg Cooperative State University, is a co-founder of the test community The TestLänd and a member of the QS-Tag program committee.
Who hasn't experienced it? The daily coffee machine, arguments in meetings or excuses when it comes to communicating the true status of a project. In software testing and test management, clear facts are more important than ever. The podcast episode with Matthias Groß and host Richie is about a test reporting framework that leads rather than brags. Matthias shows how he creates real orientation with a clear structure, automated evaluations and a focus on the essentials.
Instead of a flood of figures, Matthias uses the image of a boat trip. In a project, the weather can be good or turbulent - sometimes the journey is calm, sometimes the waves are raging. Good reporting is particularly important in rough conditions. This is the only way for the team to know where the ship is, where it is heading and whether it is going off course. This metaphor makes it immediately clear: test reporting should not only inform, but also show the way.
Matthias' approach consists of five steps that anyone can follow.
Know the goal: First of all, it must be clear: Where do you want to go? In testing, this is not only the production release, but also how much freedom from errors and test coverage is desired. You have to remain realistic; 100 percent rarely exists. The team sets the goal together - it decides what quality means.
Determine the position: Where does the project currently stand? Matthias uses a 3x3 matrix to answer this question. Three columns: Preparation, implementation, errors. Three rows: current status, progress, data control. This matrix provides a quick and clear overview.
Calculate forecast If you know the history, you can estimate what the road ahead will look like. Using the data from the last few weeks and expert knowledge, Matthias estimates: When will we reach the target? Are the schedule and progress on track? If not, there are various options - change course, strengthen the team or cut back on the goal.
Ensuring data quality: People often think that data is correct, but this is rarely the case. Matthias uses an internal control system (ICS) to check the quality of his reporting. Are there any fields that are not properly maintained, such as the sub-project? If so, this is automatically reported and the team has to make improvements. This ensures that the status in the reporting matches the actual situation.
Automation Once the structure has been defined, automation provides relief. At Matthias, the data from Jira is collected with Python, aggregated and visualized in Excel. Pivot tables provide flexible evaluations. Management receives a dashboard as a PDF, while technically inclined stakeholders use Excel for details - everyone gets the information they need.
The response to this approach has been consistently positive. Many in the project environment appreciate the overview, the focus on what is important and the ability to zoom into details. Once you have set up the matrix, you can select different views, for example by sub-project or department. The challenge? All nine fields have to be filled with data - this requires some effort, but is worth it.
Recently, Matthias has also been experimenting with AI. Traditional reporting remains stable, while the AI is used to communicate with the project data via chat. This brings flexibility and new perspectives, but does not replace structured reporting.
Test reporting doesn't have to be complicated. Those who communicate the goal clearly, present the status honestly and ensure data quality will guide the team safely through stormy times. Matthias shows how structure and automation help to avoid ambiguity and provide real guidance. Reporting that is really used and doesn't just fill a drawer is worth its weight in gold - for every project.