Minimum Viable Test Strategy

A minimum viable test strategy is a collaboratively developed, lightweight introduction to testing: not a hundred-page document, but a shared map of test levels, test types, roles, tools, environments and metrics. It creates a shared understanding, identifies open points as backlog items and immediately gives teams something to get started with.

Key Takeaways

• A minimum viable test strategy is not created through documents, but through collaborative workshops that create a common test picture with which teams can get started immediately.
• Test levels such as microservice level or product level must be redefined in the context of the architecture, because classic level models often do not fit modern system landscapes.
• The most dangerous moment in a test strategy workshop is quick agreement, not disagreement: If you agree too early, you skip gaps that become expensive later.
• Non-functional requirements such as security or performance are systematically postponed in agile projects, although the risk should be consciously decided and documented.
• A table of test levels, test types, roles, tools, environments and metrics reveals contradictions that go unnoticed when reading long test strategy documents.

What a minimum viable test strategy is

A minimum viable test strategy is a lightweight, jointly developed basic framework that teams can use to test immediately without waiting for a finished document. The term is based on the Minimum Viable Product. Instead of a formulated strategy, a starting point is created that deliberately allows for gaps and grows over time.

The difference to the classic variant is the attitude. A classic test strategy is written by one or two people over weeks, sometimes coordinated over years, and in the end nobody reads it. A minimum viable test strategy, on the other hand, provides the common understanding with which teams get started, plus a backlog for the open points.

Kathrin Potzahr summarizes the goal as follows: not the complete concept, but enough to start a continuous learning process. If nothing is there, no one can continuously improve. But you can with an initial status.

Why the classic test strategy is failing in the new world

Classic test strategies are a poor fit for modern architectures because they come from a different world. A typical case: a program is to switch from monolithic applications to microservices, cloud and agile development. The old world is well described, but nobody knows how to test the new world.

The problem already becomes apparent at the test levels. Those coming from the old way of thinking see applications as the test level first. In the microservice context, however, the service is the deployment unit and must be secured as such. A product view is also required because a product can be made up of several services.

Then there is the legacy world, which is not going away. Conversion systems that are only deployed twice a year are still attached to the new services. A good test strategy must make both worlds visible side by side instead of pretending that there is only the new one.

How test strategy storming works

Test strategy storming transfers the idea of event storming to the test strategy: a collaborative workshop with post-its instead of a solitary written document. The format is deliberately lightweight and is created in the room, not in preparation.

The core is a table as a test map. The axes are test levels and test types, the cells are filled with notes: responsible roles, tools, environments, metrics. Where knowledge is missing, a piece of paper with an open dot is added. This way, the workshop does not result in a finished concept, but a clear view of where things are stuck.

In practice, three consecutive workshops were sufficient for this. It is important not to start with ready-made cards, but with a blank table. The first question is simply: What test levels do we need? The first entry is usually a no-brainer such as unit testing, after which the actual discussion begins.

Discussion is the result, not the obstacle

Friction in the workshop is not a loss of time, but the actual value. Long discussions arise when trying to differentiate between service and product. The exact definition is of secondary importance. The important thing is that everyone involved gets a common picture of what a deployment unit is and what belongs at product level.

Even formal classifications do not have to be correct as long as the team is satisfied. Consumer-driven contract tests can be justified as a test type because they test interoperability. If the people involved prefer to run them as a test level, it stays that way. What matters is the common understanding, not the correct pigeonhole.

This is precisely where a surprising danger lies. It is more risky for a team to come to an agreement too quickly than to get bogged down endlessly.

I think it’s much more dangerous for people to agree quickly than not to agree.

Kathrin Potzahr

Why the moderator must know test strategy

A test strategy storming session needs a moderator who comes from testing themselves, because their main task is to question. Those who only lead through the agenda allow hasty agreements to pass. Those who understand the topic will prick at it: Aren’t you still missing something? Is that really the case, or is it just from the old world?

This challenge function weighs more heavily than speed. The reflex of many moderators to get through as quickly as possible misses the mark. It makes more sense to think broadly and complete the map. It will become smaller and more specific later anyway, in the individual tasks.

If a team does get lost in the small details, classic moderation techniques can help: close, stick a piece of paper with an open point and move on. The detailed discussion can be postponed without the information being lost.

The right people decide the outcome

Test strategy storming only works with the right participants, and you can’t dictate who that is from the outside. Just as with event storming, the value depends on the staffing. Agile projects often only know developers, scrum masters and product owners. Large programs have more roles that need to be involved.

There is no hard and fast rule, but there are two constants. The teams that later carry out the test work must be represented by representatives. And ideally, someone with product glasses sits at the table. In a workshop with a hundred people, the teams do not belong in their entirety, otherwise the format will collapse.

A proven line-up includes architects, test managers, operations and security experts. Depending on the context, project or program managers are added, people from the build infrastructure or, in a scaled environment, a system team. There is no single core recipe. The involvement of the teams is non-negotiable.

These components belong on the map

The test map consists of two axes and four content levels per cell. Test levels and test types span the grid, the cells are filled with roles, tools, environments and metrics.

The roles determine whether the strategy is viable. If the map itself still contains open work, it must be clear who is responsible for this work. Otherwise, the backlog remains a wish list without an addressee.

The test types are usually the more stable part because they hardly change when the architecture changes. Nevertheless, it is worth challenging them: Is robustness a separate test type or part of the functionality? Does maintainability run separately or as part of unit testing? These questions sharpen the common picture.

Metrics and target values belong at the end, not at the beginning

A minimum viable test strategy names the metric first, not the target value. It is sufficient to specify that code coverage should be measured. Whether 80, 75 or 90 percent is correct is determined later by experience.

This is not procrastination, but a method. You start with a value, observe whether there are problems and make adjustments. Perhaps 90 percent is right in one case and 75 percent in another. Without an initial level, there is no basis for precisely this improvement.

Where the requirements are not yet clear enough, for example with load and performance, the point should be marked openly. First clarify the requirements, then build the metrics from them. It is better to keep this honestly as an open point than to write down a pseudo-precision.

Non-functional requirements otherwise slip to the back of the list

A visible map protects non-functional requirements from being forgotten. In agile projects, teams focus heavily on features and push security, performance or usability to the back of the list. This does not happen out of ignorance, but out of focus.

Deliberate postponement is legitimate, unconscious postponement is dangerous. If you know that security-relevant requirements exist, you can decide: not yet critical for the MVP with a small user group, but later indispensable. Three sprints with libraries full of security gaps are manageable if the product only goes live after twenty sprints. The point is that the decision is made consciously.

This is exactly what visualization helps with. It keeps the non-functional properties present, even if they only come into play late in the test levels.

From the map to backlog items

After the workshop, the map is shown to the teams and the open points become backlog items. The large table with post-its is not glossy, but it is easy to follow. Teams can see where they are involved and what is expected of them.

Even those who were only represented in the workshop by representatives can still make requests and changes here. The teams then know their tasks. The open pieces of paper become concrete backlog items that someone has to take forward.

In one case, this was done by an architects’ guild. It took the testing topic as its task, took it to the teams and implemented it in the pilot projects. A formulated test strategy will still emerge later in more formal environments. The only hope is that it will be lived and adapted instead of ending up as a hundred unread pages.

Visualization reveals what remains hidden in the continuous text

The test map also works without a large workshop, as a tool to check existing test strategies. Long documents can be transferred to the same grid, revealing what remains invisible when reading.

An example: A project had two test strategies, one for the acceptance test and one for the stage before that. Only the transfer to the visualization made the breaks visible. Different terminology, sometimes conflicting objectives, everything that is hidden across the pages.

This is the reusable core of the method. The map forces scattered statements into a structure in which contradictions no longer slip through. Whether for a new strategy or the inventory of an old one, the picture shows more than the text.