Business-driven test automation refers to the approach of involving business experts from insurance or other specialist departments directly in test automation instead of using only IT testers. This requires low-threshold tools, adapted wording without anglicisms and continuous coaching so that experts can independently create and maintain robust, risk-based test cases.
Key Takeaways
- Technical acceptance testing can only be automated if insurance salespeople operate the tool themselves, because no tester can replace their domain knowledge.
- The acceptance testing of a customer project at Signal Iduna runs through in three minutes after automation, instead of tying up several man-days before each release.
- Technical vocabulary such as “Triple A principle” excludes specialist users; at Signal Iduna, the image “rail replacement service” replaced the terms Setup, Execute and Verify.
- Weekly ensemble testing in coaching format, where one team member drives and the others navigate, keeps automation knowledge alive in the entire team instead of in individual heads.
- Synthetic test data is the next mandatory step at Signal Iduna, because legal data separation between divisions otherwise prevents cross-divisional services from being fully tested.
Why insurers need their experts for testing
For insurance groups, technical acceptance of software is not enough. The insurance supervisory requirements for IT (VAIT) demand both technical and functional acceptance. And this is where the problem arises.
Technical acceptance requires the knowledge of insurance specialists. Testers do not have this expertise, and it cannot be acquired quickly. Training in the insurance industry takes three years, and the specific requirements also depend on the line of business.
With an all-insurer, the bandwidth becomes a factor in itself. Jörg Sievers describes the situation at Signal Iduna: around 500 applications, plus a bank and all lines of business from car to transport and health to life. No one in development knows the technical use cases of these systems in detail.
This leads to a clear consequence: the technical approvals cannot be automated during development. Those who understand the business transactions sit in the specialist department, not in the development team.
Low-threshold instead of technical: the tool must fit the target group
If specialists without an IT background are to automate, the low threshold determines success or failure. The key question is: how simple does the whole thing have to be for normal users to use it and be enthusiastic about it?
When selecting tools, there is a principle that Jörg applies consistently: The people who will later use the tool have a say in the decision. Two employees with no automation experience, but with broad technical knowledge, were given several test tools to try out and were allowed to say which one they liked best.
The tool had to be based on a familiar world. In the financial sector, everyone knows Excel, so the solution had to go in this direction. Nevertheless, pure Excel was out of the question because documentation has to be audit-proof.
Language also plays a role. Signal Iduna is a German company that is over 100 years old and has long-standing employees. English technical terms and Anglicisms do not work there as a basis for communication.
Wording changes the attitude to testing
The terminology alone determines whether experts are involved. Testing often has negative connotations along the lines of “I have to do this”. Instead of talking about testing, the focus was therefore on quality testing and test points.
Technical principles also need a clear picture. The coaches translated the Triple A principle from test automation, i.e. Arrange, Act, Assert, into the term rail replacement. Setup, execute and verify, an image that is understood just as well in Dortmund as it is in Hamburg.
The advantage of this image is practical. If someone looks at a test case and sees only two lines instead of three, they know immediately: something is missing here.
Next to it is a fixed sequence that is repeated like a prayer wheel until it sits: first get the test case running, then make it robust, then parameterize it. Only when these principles have become second nature can the teams work on their own.
Risk-based instead of “100 data sets because we’ve always had them”
Departments rarely think in terms of test objectives at first. Instead of talking about goals, the attitude is often: “We have our 100 data sets here, and these are our test cases.” When asked why, the answer is: “Because we’ve always had them.”
This is where the risk-based approach comes in. The leading question shifts to the biggest risk and the business case that covers this risk. These are usually the most complex test cases.
This results in a pragmatic rule of thumb for specialist testers: instead of working through all test cases, the most complex case that happens to cover everything else is often sufficient. Such preliminary discussions take place before the start of every project, based on a four-pillar model for the introduction of test automation.
Coaching instead of booking testers
The method thrives on close contact with the team, not on work handed in. The unit has been operating as a Center of Excellence since September. The self-image is clear: no testers that can be booked, but help for self-help.
In concrete terms, this means at least one hour of coaching per week with the whole team. The procedure is similar to ensemble testing: one person is the driver and shares their screen, the others assess whether everything is correct. The coaches take on the role of navigators and say what needs to be done.
Everyone becomes a driver in turn so that everyone has to carry out the steps themselves. Beforehand, the coaches look at the previous week’s work and collect tips on where improvements can be made.
This model works despite staff changes. One colleague only joined after the start of automation, with no prior knowledge, went through the training and coaching and did her own thing. A previously skeptical employee, who said “I’m not interested in testing”, then decided to continue in this direction.
Difficult projects first, not the harmless calculator
We deliberately started with critical systems rather than a peripheral topic. The usual approach would be to start with a small website calculator that has nothing to do with other systems. This works quickly, but doesn’t help because it remains unclear how things will work in the real world.
Instead, two important projects were chosen for the company. One automates the customer portal, which customers use to submit their documents digitally instead of in analog form. The other concerns the digital conclusion of insurance policies and other products.
The promise to the teams was specific and under pressure: once you set up the automation, you will never have to carry out the lengthy manual approval process in the same way again. Unlike external tool providers, the internal coaches could not simply claim this promise, they had to keep it.
The result after around three and a half years: No project has broken off and everyone is still as enthusiastic as they were at the beginning.
Business-Driven Test Automation: testing behavior, not going green quickly
The most dangerous reflex in test automation is to go green quickly. This is exactly what the approach that Jörg calls Business-Driven Test Automation (BDTA), based on Behavior-Driven Development, is aimed at. The behavior of the system is tested, not the shortest path to the green tick.
An example makes the difference tangible. If a user enters “20000” and Enter, the system formats “20,000.00” after a short delay and saves exactly this record in the database. Anyone who enters “20,000.00” directly in the test does not check what happens outside.
The consequence would be fatal: If a supplier breaks this formatting feature, it is not noticed in the test, while the users outside suddenly save bad data records. Therefore, the rule is: take the application as it is and don’t look for ways around it, for example past a blocked field.
We want the behavior of a system to be tested. Automators can always choose paths around the back. A field is blocked, no problem, I can get past it. That’s exactly what we don’t want.
Jörg Sievers
How two tools complement each other instead of competing
Different testing tools do not have to compete against each other, they play on different levels. In the customer portal project, the development team used Cypress, while the business side worked with Sahi Pro. Initially, this gave the impression of competition.
The solution lay in a clear division of tasks according to strengths. Cypress is well suited for a one-pager app. However, as soon as peripheral systems come into play, something is uploaded or downloaded or the user jumps to another system, the other tool comes into its own.
The tool selection was also based on a communication objective. We were looking for a simple programming language that developers could look at immediately, because practically every developer knows JavaScript. This means that the same test can be read from two perspectives.
The result of this division is measurable: the business transactions with system interaction continued to run via Sahi Pro, a large part migrated to Cypress, and the acceptance testing has since been completed in three minutes before going into production.
Test data first, then configuration, then automation
With complex systems, the sequence of steps determines efficiency. In the data-intensive customer portal project, the following therefore applied: first clarify the provision of test data, then everything else.
This resulted in a staggered sequence. The test data is generated first. Once it is ready, the system checks its configuration. Only then does the automation start.
The configuration check alone saves one person-day per release. Otherwise, if a complex system starts up with an incorrect configuration, everything starts all over again and the day is lost.
Automation can also be started in a targeted manner. If only one parameter has been changed, it is sufficient to execute the appropriate test case group in Jenkins instead of the entire run.
Helping people to help themselves means: teams develop their own ideas
A good Center of Excellence makes itself superfluous in the right places. The unit cannot do everything and therefore provides methods, procedures, techniques, tools and sometimes the test environment instead of doing the work.
This approach is illustrated by a specific task. Before a three-week absence, Jörg gave a colleague the task of solving the environment detection himself, i.e. recognizing which test system the automation is currently running on. With a few tips and without constant supervision, the colleague managed it.
Such own ideas are the actual goal. The sequence of test data, configuration check and fanned-out start was also largely developed in the teams, not as a specification from outside.
There are limits to the freedom of tools, and for good reason. Every tool used needs a technical and a functional person in charge. That’s why there is a catalog from which the teams choose, rather than a whole zoo of tools like a startup.
Automated, manual, explorative: what stays where
Not everything is automated, and that is intentional. Where there is standardized machine-to-machine communication, for example via the industry standard BiPRO in exchange with comparison portals, it can be fully automated. However, this is a different test than the one via the interface.
For internal applications without interfaces, a lot of things run via the interface and specialist departments continue to carry out manual tests. Manual testing remains the rule, especially where units hand over to specialist departments outside of agile structures.
Exploratory testing is actively encouraged. In short training sessions over two half days, the experts learn test techniques that suit the company, such as equivalence partitioning and boundary value analysis, including practical examples and the reasons for doing things this way.
Synthetic test data and contract testing as the next levers
Two topics are the next major tasks. The first is synthetic test data. As an all-risks insurer, Signal Iduna is subject to legal restrictions: Life insurance employees are not allowed to see what the health insurance company processes.
Pseudonymization only half solves this problem in testing because the restriction remains in place. Synthetic test data should release systems, also with regard to future products that overlap several lines of business. If you are not allowed to look into the other line of business, you can hardly test such a product otherwise.
The second topic is consumer-driven contract testing. The Group is transitioning from monolithic legacy systems, including WebSphere and 40-year-old Cobol code, to a distributed world of microservices. Agreement must be ensured between the services.
The appeal of contract testing lies in its self-sufficiency: teams can test for themselves without having to run the entire surrounding system zoo. This first requires projects that apply the approach and convince others.
One attitude runs through both topics. In testing, many things are solved through pain, although prophylactic action would be better. As Jörg puts it: you can work prophylactically or reactively, and the second is more expensive.
