At Otto, quality assurance in software development means more than testing: quality is built in across the whole development cycle, from the requirement to operations. Around 15 quality specialists support some 60 self-organizing teams as coaches and sparring partners. Consumer Driven Contracts secure the interfaces between teams automatically. Guilds, Quality Buddies and training carry quality awareness into every team.
Key Takeaways
- Quality is not the job of one dedicated tester. Everyone on the team has to think about it and share responsibility for it across the entire development lifecycle.
- The Quality Buddy role gives teams without a dedicated quality specialist someone who actively raises quality issues and acts as a multiplier between the team and the guild.
- At Otto, Consumer Driven Contracts are a standard task whenever a new interface between two teams is created, which automates most interface testing.
- A quality specialist’s stated goal is to become unnecessary: the job is done only when the team has the same know-how and mindset.
From a Separate Test Team to Quality Assurance Inside the Team
Otto disbanded its classic test team and moved responsibility for quality assurance into the development teams. The company used to work in a waterfall style: a separate test team waited for development to deliver the software, then ran manual functional tests and reported back.
The change started when Otto began building and running its online shop, and later its app, entirely in-house. With in-house development, the mindset shifted, and the separate test team was folded into the development teams. At first the testers kept doing their usual testing work, but now they sat right in the team, with shorter feedback loops.
That first step was not the destination. Dominique Petrich, Quality Manager at Otto, describes a different target: away from classic ISTQB roles such as tester and test manager, toward sparring partners, communicators, moderators and coaches within the team.
Quality Comes from the Whole Lifecycle, Not Just Testing
Quality has to be considered across the entire lifecycle of a requirement, from concept through go-live and into operations. Testing is only one part of it. The actual goal is broader: processes, the team and the way people work together are part of it too.
That leads to an unusual definition of the quality role. A quality manager works toward becoming unnecessary in the team.
“I’m done when my colleagues on the team have the same know-how as I do, the same mindset, the same awareness of quality.”
(Dominique Petrich)
The role name reflects this shift. What used to be the department’s Quality Specialist later became the Technical Quality Manager, mainly to have one consistent title across the company.
Why Team Ownership Carried the Transformation
The agile transformation at Otto was not ordered from the top. Everyone involved helped shape it. That raises acceptance, because nobody gets someone else’s process forced on them.
With in-house development, Otto set up its teams to be cross-functional. The idea: in principle, everyone on the team should be able to think along and work on everything. Improvement never stops. Just as software is built iteratively, a team develops iteratively too.
Ownership comes with obligations. But the teams want to own that responsibility and get better. That drive to improve is exactly what makes quality topics come from inside the team instead of being imposed from outside.
How Testing Know-How Is Spread Across the Team
The first lever was spreading testing know-how across everyone. Instead of one QA person designing, writing and running tests alone, everyone on the team learns to do it. Otherwise a single testing resource becomes a bottleneck.
The first topics are hands-on: how do I write tests, with which frameworks, how do I run them? From there, the team works its way step by step through the lifecycle.
Otto doesn’t prescribe a fixed tech stack. The teams are largely autonomous and choose their own programming languages, frameworks and tools. Only certain things are fixed and organized centrally.
What the Workflow Looks Like in the Teams
Otto works with a mix of Scrum and Kanban that people there call Scrumban. The teams picked the practices that suit them instead of copying a framework one to one.
In Dominique’s team, work runs through a board with clear stages: analysis, development, acceptance, go-live and a follow-up review. Stories can enter the prioritized backlog at any time and get worked through. There are no dedicated sprint plannings with fixed up-front commitments.
The team still estimates, but for a different reason. Estimating only serves to surface where discussion is needed. If estimates are far apart, team members understand the story differently and need to talk again. The estimates are not recorded in the stories and are not used as a measure of success.
The two-week rhythm stays in place to keep the teams in sync and to break down higher-level plans, from the company to the division to the individual team.
A Quality Guild Against Lost Synergies
With around 60 teams in e-commerce, there is a risk: synergies get lost, and quality becomes reactive instead of preventive. A process gets set up, runs and is never touched again, and only after a major failure does someone look at the causes.
Preventive quality thinking is meant to stop exactly that. In response, four colleagues founded the Quality Guild, a regular exchange for everyone in the division who works on quality. It covers current challenges, mutual support, interfaces and learning from each other.
The numbers show why a cross-team format is needed: about 15 quality specialists for roughly 60 teams. So not every team has a dedicated QA person, and that is a deliberate choice.
Quality Buddy and Security Champion: Roles Anyone Can Take On
For teams without a dedicated QA person, Otto created the Quality Buddy role. This person champions quality in the team, raises the topic when something looks off and acts as a multiplier between the team and the guild.
The role isn’t tied to a job title. A Quality Buddy can be a developer, a domain expert or someone from UX, regardless of their actual role.
The same pattern repeats for security. Every team has a Security Champion who takes part in a separate exchange format and anchors security topics more firmly in the team.
On top of that come concrete offerings: two workshops in which teams work on their own quality topics, a training program for new colleagues and occasional talks, some with external speakers.
Why the Basics Need Repeating
Even experienced developers benefit from refreshing testing basics regularly. Onboarding often assumes that new colleagues bring a certain mindset and knowledge. It still pays to go over the basics again.
That includes equivalence partitioning, decision trees and writing clean test cases. Most people have heard of these concepts, but they fade into the background in daily work. The feedback is clear: it was good to hear this again.
How Otto Tests Between Teams: Consumer Driven Contracts
For testing between two teams, Otto uses Consumer Driven Contracts, or CDC. These automated interface tests run in the CI/CD pipelines of the teams involved.
By now this is simply how things are done. Anyone who builds a new interface to another team gets the automated interface test as a standard task in the backlog. That gives Otto a very high degree of automation across one or a few teams.
Beyond two teams it gets harder. As soon as end-to-end testing across several division boundaries is needed, manual work and communication go up sharply. There is no ready-made solution for that yet, and Otto sees it as one of its biggest open challenges.
Covering Non-Functional Requirements Through Specialized Teams
Otto covers security, load, performance and availability through dedicated teams, combined with roles in the product teams. Security has its own team, with mandatory training for everyone.
For technical safeguards, Otto relies on the tooling of the platforms it uses. Since all repositories live on GitHub, GitHub services are mandatory for every team. The infrastructure on Amazon’s cloud runs automated scans that alert teams to anything unusual.
For load and performance, a dedicated team runs cross-team tests against the environment on a regular schedule. If something shows up, that team contacts the affected product teams. Those teams can also work with the load testing experts to design their own tests for their services.
The shop must not go down, or it gets expensive. That is why alerting and monitoring are each team’s own responsibility.
Disaster Recovery as a Recurring Drill
Once a year, Otto holds a Disaster Recovery Day. An emergency is simulated, for example a compromised AWS account that forces a complete rebuild. The goal is to restore all interfaces as fast as possible, highly automated and together with other teams.
These days show where the gaps still are. Meanwhile, some teams also run such drills on their own on a regular basis, to automate the process further and get faster for a real emergency.
Usability Through UX Roles, A/B Testing and Real Customer Feedback
Otto secures usability through UX specialists who sit directly in the front-end teams. UX designers and UX managers bring usability and accessibility requirements into the team, whether for the app or the web shop.
On top of that comes systematic customer feedback. Otto runs regular surveys and A/B tests, rolling out features to only part of the customers at first. Both get measured: how the features perform and what the feedback says.
On its campus in Hamburg, Otto runs its own UX lab. Real customers are invited in, look at features on screen, are tracked and say right away whether they could use something intuitively.
Static Analysis Is Up to Each Team
There is no company-wide standard for static analysis at Otto. Some teams use it heavily, others hardly at all. That, too, reflects team autonomy.
In practice, some teams use linting and check test coverage more locally, for example while pairing or through the QA person. Other teams build quality gates into their CI/CD pipelines, set a minimum test coverage and run automated analyses against it.
That range is intentional. Instead of mandating static analysis from outside, the decision stays with the team.
The Biggest Open Challenges for the Coming Years
End-to-end testing across several divisions remains the biggest technical challenge. Large projects that touch the shop, the backend and several areas should be tested with as little manual effort as possible and as part of the agile process. That topic will stay with Otto for a while.
AI is also becoming part of everyday work. Otto already works a lot with algorithms, is trying out how AI can support programming, for instance when writing tests, and is collecting best practices.
New channels create testing needs as well. In the future, people will shop not only through the web shop and the app but also through a smart TV or the car, for example. Every new channel has to be integrated, tracked and tested.
The biggest structural trend concerns the QA role itself. Going forward, the number of QA specialists won’t grow with the number of teams. The goal is for teams to own quality completely.
That moves the quality manager’s job up a level. Instead of moving from team to team, they move into cross-team work: creating offerings, helping teams become self-sufficient and driving quality awareness across teams and divisions.
Frequently Asked Questions
Why Is a Single Tester Not Enough in an Agile Team?
A single testing resource becomes a bottleneck. Instead of having one QA person design, write, and run tests alone, everyone on the team learns to do this work. The topics start with the basics: how to write tests, which frameworks to use, and how to execute them. From there, the team gradually works its way through the lifecycle, from requirements all the way to production.
What happens to traditional testing roles, such as testers and test managers, in an agile organization?
At Otto, the separate testing team was dissolved, and the testers joined the development teams with shorter feedback loops. The vision moves away from traditional ISTQB roles toward sparring partners, communicators, moderators, and coaches. The Quality Manager works explicitly toward making themselves redundant: their job is done when the team possesses the same expertise and mindset.
How can quality be ensured if not every team has a QA specialist?
Through the role of the Quality Buddy. With about 15 Quality Specialists and roughly 60 teams in the e-commerce division, many teams do not have a dedicated QA person, and this is a conscious choice. The Quality Buddy raises quality issues when anomalies are detected and acts as a liaison between the team and the Quality Guild. The role is not tied to a specific profession: developers, subject matter experts, or UX professionals can take it on.
How do you test interfaces between teams without manually coordinating everything?
With Consumer-Driven Contracts that run as automated interface testing in the CI/CD pipelines of the involved teams. At Otto, the test is a standard task in the backlog as soon as a new interface with another team is created. This results in a high degree of automation across individual or a small number of teams. Beyond two teams, the manual effort increases significantly.
Why should teams still estimate stories if there are no fixed sprint commitments?
At Otto, estimation serves only to identify areas that need discussion. If the estimates vary widely, team members have different understandings of the story and discuss it again. The numbers do not appear in the stories and are not considered a measure of success. The two-week cycle remains in place, however, to synchronize teams and break down higher-level plans.
Who is responsible for non-functional requirements such as load, performance, and security in autonomous teams?
Otto provides coverage for these through dedicated teams, combined with roles within the product teams. A separate security team enforces mandatory training for everyone, and each team also has a security champion. Another team runs cross-team load and performance tests on the environment and reaches out to the affected product teams if any anomalies are detected. Alerting and monitoring remain the responsibility of each team.
How do you verify that an emergency and recovery process actually works?
By simulating an emergency. Once a year, Otto hosts a Disaster Recovery Day, for example, by simulating a compromised AWS account that forces a complete rebuild. The goal is to quickly restore all interfaces in a highly automated manner and in collaboration with other teams. Such days reveal gaps, so individual teams also conduct additional drills on their own initiative.
Should static code analysis be mandated uniformly across the entire company?
No. At Otto, there is no uniform standard for this; the decision is left up to the team. Some teams use linting and check test coverage more locally, during pair programming or through the QA person. Others build quality gates into their CI/CD pipelines, define a minimum test coverage threshold, and run automated analyses. This range of approaches is intentional.
