Exploratory ensemble testing encourages teams to reconnaissance software together and without rigid test cases. The method thrives on changing roles, clear processes and short test intervals. The focus is not purely on finding errors, but on learning about the product and gathering a variety of perspectives, whereby usability issues also come to light. The approach promotes testing competence, team dynamics and leaves room for new ideas. Regular meetings help to capture feedback at an early stage and initiate improvements. Those who are prepared to open up structures and see testing as a joint learning process receive valuable impetus for the further development of software quality.
In this episode, I talk to Tobias Geyer about exploratory ensemble testing. He has been using the method for two years and shows why it works. We discuss how the Driver and Navigator roles work, the rotation of roles and the compact 90-minute sessions. The result: early feedback, lots of usability findings and, above all, learning as a team. Developers discover how to look beyond the happy path. Disputes? Hardly, because we collect notes and do the evaluation later. Clear charters, good moderation and disciplined note-taking are important.
"And that really helps to get this feedback earlier." - Tobias Geyer
Tobias Geyer never consciously wanted to become a tester, but after more than 20 years of experience in this profession, he is happy to have become one.
His experience ranges from waterfall processes in large companies to small agile teams. He always keeps an eye on the quality of the overall product and takes care of all issues that contribute to this, whether it's improving CI systems, creating test cases or introducing test methods. His enthusiasm for learning and sharing knowledge led him to co-found the "Software Testing User Group Hamburg". He loves speaking at conferences and networking with people from the community.
Testing together produces different results than testing alone in front of a screen. For many, this is unfamiliar at first. In the podcast "Software Testing", Tobias Geyer gave an insight into how his team has been using explorative ensemble testing in everyday life for two years.
It was previously known as mob testing. But the term mob has negative connotations, which is why "ensemble testing" has become established. The concept is simple: a group sits together in front of a computer and tests software together. There are fixed roles: Driver operates the computer, but is not allowed to make any decisions himself. The navigator makes the decisions and tells the driver what to do. The rest - the ensemble - observes and contributes ideas. The roles change regularly. In addition, there is the principle of exploratory testing: there are no rigid test cases, but a charter, i.e. a topic or goal. The group comes up with the tests based on what they find.
How do teams come to work this way? Tobias Geyer learned about the method at a conference. At first, he was skeptical as to whether it would work in his team. Many developers found testing boring. But then there was a problem: errors often only appeared in late test phases, often too late. Requests or bugs from stakeholders often only appeared shortly before the release. This frustrated everyone.
The solution was an experiment. "We'll just give it a try," thought Tobias Geyer. The invitation went out to the whole team. Anyone who wanted to came along. Since then, there has been a session with a new charter every four weeks. With success: bugs are found earlier and the developers' understanding of testing is also growing.
A session usually lasts 90 minutes. At the beginning, we discuss what will be tested. Then we get started. After every 8 to 10 minutes, the roles change. So everyone gets a turn at the wheel - and sets the direction. "That's sometimes difficult," says Tobias Geyer, "because some people would like to do everything." But that's exactly the point: everyone has their say, no one decides everything.
Anyone from the team can take part, sometimes even product managers or other departments. This brings together a wide variety of perspectives.
Two years later, one thing is clear: it pays off. "We have found pretty much everything - from simple errors to things that nobody would have thought of," says Tobias Geyer. Particularly important: many usability problems were uncovered. "If someone suddenly moves a text using drag & drop and the code crashes - no one would have thought of that on their own, but together someone comes up with the idea."
Another result: developers learn how to test in a meaningful way. They also transfer their knowledge to their daily work. Tobias Geyer is pleased with how the methodology sharpens his view of the system as a whole. And the sessions are even fun. Getting out of the daily routine, seeing something new, being creative as a team.
Of course, not everything goes perfectly. The biggest challenges are:
Taking notes. It's easy to forget what you've found. That's why someone should always take notes.
Keep to the roles. Curiosity often wins out, but that's the only way it works fairly.
Motivation. After many sessions, routine can set in. Changing topics helps here - always different features, different ways of using them, inviting new people to the session.
Over time, the group sometimes shrinks and every now and then a session has to be canceled. But the positive feedback remains.
In the end, Tobias Geyer advises: "Try it out. All it takes is an invitation and a bit of courage. Finding bugs is not the main goal - it's learning together that makes the software better. Testing can be fun, and new ideas are created together, not alone. Once you have worked in an ensemble, you see software with different eyes.