Quality coaching in the SAFe environment refers to the coordination of development and testing activities beyond the boundaries of individual teams. SAFe (Scaled Agile Framework) connects several teams in an Agile Release Train, but leaves open a role that controls overarching quality. Quality Coaching fills this gap through requirements coaching, process workshops and defect management at all levels.
Key Takeaways
- Those who switch directly from waterfall to SAFe bring legacy issues with them: fixed target deadlines without a clear scope, where functionality is reduced until the deadline fits, which systematically undermines quality.
- Development teams that only know their own story cannot ensure overarching quality: Errors occur because no one knows how their component is embedded in the overall business process.
- A physical process simulation, in which testers from different teams play through their dependencies together, creates a shared understanding of the product faster than any document.
- Bugs at an overarching level remain unresolved because no team feels responsible: a designated bug coordinator per release train is the pragmatic solution until teams take on this responsibility themselves.
- Negative test cases and edge cases require specialist knowledge that is often not fully available at any level in SAFe transformations and is hardly ever efficiently transferred from the solution level to the development teams.
What is Quality Coaching in SAFe
Quality coaching is a role that coordinates quality responsibility across the boundaries of individual development teams. It closes a gap that arises when agile scaling frameworks such as SAFe bring multiple teams together on a common release and a common system.
In agile working, quality is the responsibility of the development team. Everyone in the team contributes to it. This works well in a single scrum team as long as the team works in its own bubble.
SAFe adds another level above the development team: the agile release train. It combines the work of several teams towards a release. This means that the development results of the individual teams must fit together, and it is precisely this fit that must be tested. No one is responsible for this coordination in the standard set-up.
Quality coaching is not just test coordination. Depending on the situation, it is also communication coaching, requirements coaching and method coaching. All aspects that together ensure that a usable quality is achieved in the end.
Why the quality gap arises after a transformation
The gap arises because teams in the transformation from waterfall to agility bring with them an understanding of work that is limited to their own piece. Those who have worked in waterfall for years are used to working through their test cases and being done when their own results are right.
After the changeover, a tester often still understands that they are testing their story and reporting it as finished in the review. What comes afterwards is lost sight of: the feature that has to be rolled out to the system together with the stories of other teams.
This agreement between teams on the same feature was not established. It was no longer known that it had to be done at all. As a result, variables were assigned differently at the development level and validations worked differently at the functional level. This led to contradictions in the same system.
Many of those involved are simply overwhelmed in this situation. They do not know what they need to do to ensure that the overall result works. Even one simple question remains unanswered: If ten development teams work in an Agile release train, who provides the test environment on which all ten teams test? Each team initially says it is only responsible for its own tests.
How to analyze the quality gap
Every intervention starts with analysis. Before you change anything, you need to understand how the teams work and why certain defects occur at product level.
It helps to sort out the causes of errors. Are the errors due to the methodical approach? Is it communication because A doesn’t talk to B? Or are they architectural errors because something went wrong at the planning stage because the full scope was not known?
A structured interview based on a questionnaire makes the picture more reliable. It is conducted across all roles and levels: Testers in the development teams, scrum masters, product owners, product managers and engineers at higher levels. The key question in each case is: How do you understand your responsibility and your influence with regard to the overall product?
The result of such interviews is often sobering. At development team level, the product is sometimes completely unknown. A team knows that it should adapt an interface to include an attribute, but not what is transferred via the interface and how this is embedded in the business process. In some cases, this lack of knowledge extends all the way up to the level of the Agile Release Train, which means that no better guidance can come from there.
Simulate business processes instead of explaining them
An effective format for creating a comprehensive understanding is the physical simulation of business processes. Testers from the development teams involved run through the complete processes in a room, with paper on the wall.
Everyone takes on a role in the process. One is the ID generator and creates an ID. The next person says they need an ID, searches for data and goes to their database. The database returns the requested data. In this way, the entire process is played out station by station.
This simulation is effective because it answers questions that remain unanswered in day-to-day work: Who am I working with? What does the person in front of me do, what does the person behind me do? What exactly does he need to be able to work? And what is the added value of the whole thing for the company?
The result is a unique identity within the overall construct. Once you have gone through the process, you look left and right in development and testing. This understanding results in joint tests across several teams: a tester then says that he wants to check that the ID and data work, that he needs three other teams to do this, and makes an appointment.
Requirements must be explainable, otherwise they will be rejected
A requirement that cannot be explained clearly in a short space of time is not yet ready for implementation. A filter can be built from this simple rule.
The rule of thumb in coaching: if an epic, a feature or a story cannot be explained in ten minutes in such a way that all critical questions are clarified in the five minutes afterwards, the whole thing will be rejected. Then it’s simply not ready for someone to work with it in a meaningful way.
Definition of Done and Definition of Ready are a prerequisite in agile working and control the quality that comes out. In a company that decides on SAFe at short notice without thinking it through properly, these artifacts are not there by themselves. They first have to be established.
A team that has only limited knowledge of how SAFe works cannot be set to this task without guidance. Otherwise, work results are produced for the sake of results, not for the sake of quality.
Who is responsible for the shared test environment?
In an agile setup, responsibility for the overarching test environment belongs to the system architect. However, the first and most important step is to recognize that this responsibility does not lie with anyone.
At team level, the lead can be distributed pragmatically. The team that already operates the most comprehensive test environment with the most connections to other services and teams makes sense. It takes the lead and the other teams deliver. This places responsibility where it belongs in the agile model: with the development team.
However, there is still a problem at the boundaries of responsibility. As long as not every delivery is clearly defined, not everything fits together. This is where the system architect comes back into play, who has to sit down with all interface partners to coordinate the parts.
The mix of the system landscape makes things more difficult. New microservices stand alongside legacy monoliths with completely different release cycles. An environment in which everyone can test together is still a big step compared to the initial state, in which there was none at all.
The target picture continues: if a service releases a new version, this should be automatically installed on the environment and the service itself proactively reports that there is something new. Today, release notes are often run through instead, from which it is first necessary to read out what belongs on the test environment. Time-consuming, but functional.
Defect management needs a clear coordinator
Errors that occur in an overarching area need a designated coordinator because otherwise no one feels responsible. In an overarching room, no one was responsible and no one wants to be.
The error coordinator initially accepts all errors for an agile release train or solution. He checks which area an error falls into, talks to the person responsible for this area and the person responsible calls in the appropriate development team. Errors are therefore collected centrally and distributed in a coordinated manner at working level.
When distributing to several teams, several paths exist in parallel. A defect is assigned one after the other, or it is assigned tasks that each team works on individually, or it is cloned for each team. This is where the responsibility of the quality coach deliberately ends, as interfering with the agile way of working is detrimental to the self-organization of the teams.
A standardized process is recommended so that nothing slips through the cracks. This recommendation is taken into retrospectives, but is often acknowledged by the teams with the comment that nothing slips through. An attitude close to that of the developer who says he doesn’t make mistakes.
Professionalism is the bigger gap than technology
The biggest open weakness is not in technical testing, but in technical understanding across team boundaries. At the team level, component testing and integration testing work well. Problems occur at the overarching level.
In the end-to-end test chain, the technical understanding of the application is often still there, but the business understanding is no longer. A tester is then technically able to try out all combinations of an interface, but does not know for validation purposes whether a particular case is allowed to go through at all.
People only see what is in the story. If the overall business process does not state that certain things are not allowed to work, then they don’t know that, then they test it and it’s okay.
Andreas Neumeister
As long as the technicality is only known to a limited extent, negative test cases and edge cases are left out. If you don’t know what is technically not allowed to happen, you can’t test it. The happy path is tested, the rest fails.
There is also a lack of an efficient channel for transferring technical knowledge from the solution level to the individual development teams via the agile release train. If a solution manager speaks to 20 or 25 teams, each piece of content is only relevant for a subset. The unaffected teams switch off, and nobody has time to tell the same thing 25 times individually.
Target dates from the waterfall poison quality
Waterfall target dates, which are carried over unchanged into agile working methods, are one of the most damaging legacies of a transformation. In the waterfall, it was easier to plan for fixed deadlines. These deadlines were carried over into agile, but now with an unclear scope because the work is agile.
The result is a contradictory specification: fixed deadline, open scope. In order to meet the deadline, the functionality is tweaked until it fits. The target date is met, but you no longer have the application you wanted.
This approach is poison for quality assurance. Functionality is constantly taken out, put back in, reprioritized, and then at some point it’s a case of more functionality and fewer tests. Such legacy issues make working in quality assurance unnecessarily difficult.
The actual aim of quality coaching is to make itself superfluous. The basic agile idea that responsibility for quality lies with the development team is only fulfilled when everyone accepts this responsibility, implements it and knows how to do it. And not just the tester, but also the requirements engineer and the developer, supported by a shared understanding of quality and common quality goals.
