Building up testing for telecommunications software means developing functional test cases, load tests, and in-house automation tools step by step, often without any ready-made solution on the market. External requirements from partners such as Cisco provide concrete exit criteria. Internally, ISTQB certification and regular team reviews keep knowledge transfer and quality on track over the long term.
Key Takeaways
- With no market solution for their specialized load tests, Bucher + Suter built their own test automation software from scratch, because no available tool covered the combination of voice gateway control and CRM simulation.
- Cisco requirements such as the 99.99 percent target for correctly handled calls taught the team that the same numbers can say different things depending on how you phrase the failure case.
- At Bucher + Suter, ISTQB certification up to Advanced Level gives the team a shared vocabulary and makes sure even experienced testers regularly check their knowledge.
- The move to the cloud makes test automation in CI/CD pipelines more important and brings new IT security and data privacy requirements that dedicated on-premises environments did not have in this form.
When There Is No Test Basis, Testing Starts with Research
What does ISTQB in practice look like when the systems you test come without documentation? Undocumented third-party systems push testers into exploratory work. If you test interfaces between unfamiliar platforms, nobody hands you a finished list of requirements to work through. You have to figure out the expected behavior yourself, one step at a time.
That was exactly the starting point for telecommunications solutions that sit between the Cisco world and large CRM vendors. Some of the systems to be connected were unknown, and their proprietary quirks were barely documented. The expected results, meaning how the system should respond to a particular call or transaction, could not be derived from any specification.
Alexander Meister describes building such a test catalog as research. Use cases took shape through repeated rounds with experts from the platform involved, often two or three times a month. Over time, that process produced a catalog of 400 to 500 test cases, which were then adapted to the different platforms.
This kind of test design takes more thinking than ticking off predefined criteria. You don’t just look for defects. First you define what correct behavior would even be.
Moving from Developer to Tester Is a Change of Mindset
Switching from development to testing means flipping your perspective. A developer is proud of what their software can do. A tester is interested in the opposite.
“What you can do is all well and good, but I’m interested in everything that doesn’t work well. I’m interested in what you can’t do.”
(Alexander Meister)
That shift is hard at first. The point is to look for faults in the software, not in people. Whether it comes down to character or to something you grow into is an open question. Either way, it is an attitude you have to build. It doesn’t come automatically with the job title.
Why There Was No Off-the-Shelf Tool for Load Testing
Systems close to the hardware can’t be tested with standard load tools. Once tests go deep into the layers, directly at voice gateways for example, there is no user interface a generic tool could drive. The test has to start at the API level.
That is what made setting up the load test environment so demanding. On one side, load generators simulated the voice load including the full voice stream, meaning real traffic over the line rather than simulated signaling alone. On the other side, the team needed something that abstracted the agents of a call center and the CRM world behind them.
That counterpart wasn’t for sale. It had to be built from scratch as separate automation software. If you want to run load tests in an environment this specialized, you often end up building the test robots yourself.
One condition is non-negotiable: the test infrastructure has to be more stable than the system it is meant to test.
“You can’t test anything if the software you build around it breaks down first.”
(Alexander Meister)
Why a Load Test Environment Has to Be Isolated
Shared infrastructure skews long-running load tests. If you start a 48-hour run on an environment that is also open for other uses, you risk aborts for reasons that have nothing to do with the test.
A telling example: a test has been running for 46 hours, then someone pulls an update over an open network share, and the run is dead. You start over, without there ever having been a real defect.
The consequence is a dedicated load system. Its own core switch, its own servers, a rack used for nothing else, closed off from outside, with access only through a console. Only that isolation makes sure a failed test points to a real fault and not to interference from outside.
Fixed Exit Criteria Force Precision
For OEM-labeled products, the customer sets the test criteria. The exit criteria were fixed because the product ran under someone else’s label. The milestones to hit were not up for negotiation. They were a condition of delivery.
Requirements like these lead to weekly acceptance tests that show what works and what doesn’t. They also force precise language. One example: 99.99 percent of calls must be handled correctly. Is that the same as saying 0.01 percent may fail?
At ten million calls, the difference adds up quickly. What sounds like hair-splitting at first becomes a real gap at high volumes. Discussions like these shape how a test team reads and writes criteria.
Quality Stays a Team Responsibility, Even with Test Specialists
Testing doesn’t scale as a one-person show. What started with one person in 2006 grew early on: the first employee joined in 2007, and today nine people work on testing and quality assurance.
The importance of quality assurance was clear from the start, and so was the effort it takes. Quality assurance doesn’t happen by itself, and it never runs on autopilot. That early insight carried the growth of the team.
One thing matters here: there are test specialists, but responsibility for quality sits with the whole team. Specialization doesn’t replace shared responsibility. It adds to it.
How Knowledge Transfer Is Organized in a Test Team
Knowledge flows best through fixed formats, not by chance. The test team runs two rhythms side by side.
- Once a month, the whole team: The team digs into testing topics, such as problems from the current month or new software that might be worth a look. Everything related to testing goes on the table.
- One-on-ones every two weeks: The QA lead sits down with each person in turn to sort out project-specific topics. Anything that comes up there can go on the agenda for the next team meeting.
That keeps information moving across project teams. Responsibilities and the matching expertise are assigned to specific people instead of floating around vaguely.
ISTQB in Practice: Certification Creates a Shared Language
ISTQB certification mainly serves one purpose: a shared vocabulary. When everyone means the same thing by a term, testers don’t talk past each other. The Foundation Level is the base, and new team members either bring it with them or catch up on it.
The benefit runs both ways. New people start with a common set of concepts. Those who have been around longer stay sharp through regular contact with freshly certified colleagues.
The higher levels build on that, depending on focus. Someone heading toward test management takes a different path than someone focused on technical testing or test analysis. The goal is to certify people up to Advanced Level, so they carry that foundation with them wherever they go.
Beyond building expertise, contact with the testing world outside your own company matters too. Those connections create an exchange you couldn’t get internally.
The Cloud Shifts the Focus to Automation and Data Privacy
Moving to the cloud takes test automation to a new level. With CI/CD, building clean pipelines becomes central, so tests run reliably and keep things moving. Automation gains weight because it becomes a prerequisite for delivery.
At the same time, IT security and data privacy move to the front. In the cloud, these questions look different than for a system running on a dedicated server of your own. If you move data and services, you have to rethink security and protection.
Automation and security are the two big topics right now. They follow a pattern that runs through the whole setup: every new platform brings its own challenges, and testing grows along with them.
Frequently Asked Questions
How are test cases created when there is no documentation available for the third-party system to be integrated?
They are developed through exploratory testing. Without specifications, it is impossible to determine how the system should respond to a specific call. In the project described, the use cases were developed through repeated consultations with experts from the involved platform, two to three times a month. This resulted in a catalog of 400 to 500 test cases, which was subsequently adapted to the individual platforms.
What kind of transition can developers expect when they switch to testing?
The perspective shifts completely. A developer takes pride in what their software can do, while a tester is interested in exactly the opposite: everything that doesn’t work. This includes looking for errors in the software, not in people. This mindset doesn’t come automatically with the job; it must be learned.
Can voice-based systems be tested with generic load testing tools?
No. When tests are performed deep within the system layers, for example directly at voice gateways, there is no interface that a standard tool could interact with. The test must be performed at the API level. Load generators produce the voice load along with the complete voice stream, while the other side, which abstracts the call center agents and the CRM environment, had to be built as separate automation software.
What happens if long-term load testing runs on shared infrastructure?
It aborts for unrelated reasons. A real-world example: A 48-hour run lasted 46 hours, then someone downloaded an update via an open network share, and the test was ruined, even though there was no actual defect involved. The result is a dedicated load testing system with its own core switch, its own servers, and access only via console.
Why is the precise wording of exit criteria such as 99.99 percent so tricky?
Because the same numbers can convey different meanings. The requirement that 99.99 percent of calls be handled correctly is not necessarily the same as saying that 0.01 percent may fail. With ten million calls, what seems like hair-splitting becomes a tangible difference. Discussions like these shape how a team interprets and defines criteria.
How can knowledge transfer be organized within a test team?
Through regular meetings rather than random conversations. Once a month, the entire team meets to address test-specific topics: issues from the current month, new software, and everything related to testing. Every two weeks, one-on-one meetings with the QA manager are held to address project-specific questions. Points raised during these one-on-one meetings are brought up at the next team meeting if necessary.
What benefits do ISTQB certifications offer a team that’s already experienced?
Above all, a shared vocabulary so that testers don’t talk past each other when using technical terms. The Foundation Level serves as the foundation, upon which different tracks build depending on the focus, such as test management, technical testing, or test analysis. Even experienced testers can check their knowledge through interaction with newly certified colleagues. Added to this is the opportunity to exchange ideas with testers outside their own company.
How does the shift to the cloud change testing?
Test automation is becoming a prerequisite for deployment. With CI/CD, the focus shifts to setting up clean pipelines so that tests run reliably. At the same time, security and data privacy are gaining importance, because these issues are handled differently when data and services are hosted in the cloud compared to a system running on a dedicated, on-premises server.
