Enterprise testing refers to the testing of purchased, configurable large-scale systems such as SAP, where the focus is not on local changes but on their effects on dependent processes and interfaces. The biggest challenge lies in collaboration: testers are usually experienced professionals with no test craft, and configuration changes can generate far-reaching side effects that are difficult to predict.
Key Takeaways
- Configuration changes in enterprise systems such as SAP have a system-wide impact because many parameters are interdependent. Local functional testing is therefore never sufficient.
- Specialist testers without test training almost exclusively test the happy path because they have learned to work around problems in their professional lives instead of actively looking for them.
- Evaluating defects according to categories, whether training gaps, missing rights, functional errors discovered too late or genuine integration errors, generates more learning effect than any abstract retro.
- Management-ready visualizations of test coverage, even if the underlying metric is weak, create the attention that gives teams more time for testing.
- Experienced business users are the strongest resource for exploratory testing in the enterprise environment because they have years of system knowledge and a vested interest in how the software works.
What distinguishes enterprise testing from normal software testing
Enterprise testing revolves around purchased systems that have been customized for a company and are intended to map its processes. SAP is the best-known example, but the genre is larger: specialist systems for social welfare or other large organizations also function according to this pattern. Preconfigured system, huge user base, local adaptations.
The key difference lies not in the local change, but in its effect. The functional testing of a single customization is usually irrelevant because the developer or configurator covers this well themselves. It becomes interesting as soon as you look to the left and right: How does this parameter affect the rest of the system?
Ursula Beiersdorf puts it in a nutshell: SAP is like a large global variable in which many things are connected to many things. It is precisely this effect of a configuration change that is the actual test object. The complexity also lies in the interfaces to and from SAP, i.e. in dependencies that a classic agile team does not have in this form.
A common misconception is that standard software has already been tested, so a few adjustments are enough. This is often true for the software as a product, but the possibilities for intervention and external dependencies create side effects that emerge late in the process. Anyone who dismisses configuration as harmless is underestimating it.
Why the true complexity lies in collaboration
The biggest problem in enterprise testing is not the technology, but the coordination. “I get the people around the table who need to have a say” describes the core of the daily work better than any test method.
This dependence on many participants shifts the role. Anyone who wants to ensure quality in a large system takes on more governance tasks than in smaller projects. It is a matter of specifying narrow, practicable rules that are orchestrated centrally without implementing every step centrally.
Who tests in an enterprise environment and why this is a challenge
In enterprise testing, there are usually no trained testers. Instead, testers are specialists who know the system as users but have little knowledge of testing. They have many other tasks and test on the side.
These experienced users are strong in experience-based testing. Anyone who has used a system for ten years knows where the bad spots are and looks there specifically. As long as changes remain small, this works. For larger changes, these users lack the tools and have to work their way through them without help.
The first step in empowerment is not methodology, but a different mindset. Professionals have learned to walk around the puddle in business. Testing means jumping in with both feet and being happy about it.
Just basically thinking what can break is not the mindset you have when you learn to walk around the puddle in business.
Ursula Beiersdorf
The most effective way is to start with the person’s real user story. She writes her test, then we take it apart together. Two patterns almost always emerge: the acceptance criteria are never detailed enough, and only the happy path is tested. Only when the person has experienced this themselves is it worth starting with design methods. This is usually three steps later.
Empowerment works via staggered packages, not via a firework display of methods
Instead of imposing the full methodology on everyone, it helps to reduce the complexity of the approach. A team that supports others can tailor its offering like products.
- Small package: for people who only need to test a single user story.
- Larger package: for serious integrative testing, including test management and tool usage.
- Optimizing version: Pipelines and advanced automation for those who need it.
Many people start with the small package and quickly realize that they won’t get very far with it. More important than any package is to meet people where they are. The work of persuasion, the why, is organizational change and therefore the biggest part of the work. Methods only come afterwards, and where an area is really critical, you specifically bring in a tester or consultant who can do it.
How to successfully switch from Excel to a test management tool
Excel has long dominated the enterprise environment, and a hard cut seems counter-intuitive at first. The switch is successful if it is linked to a transformation that is already underway. When the use of Jira became mandatory, the introduction of X-Ray could be carried out at the same time.
Existing software that will be phased out in the next few years can stay with Excel. The new tool is mandatory where it counts. From this mandatory part, the first users learn to appreciate the benefits and come voluntarily: storing test cases in a structured way, recognizing their own status, having tests executed by different people. These test management functions make many things easier, and word gets around.
Defect management: four categories say more than twenty-five
Not every defect needs a ticket. Small defects that can be solved directly at the “Hey-Joe level” with a known IT colleague may be solved in this way. As soon as an error cannot be solved immediately or it is unclear who is responsible, it should be written down as a defect.
The real insight is gained during the evaluation. A simple division into four categories is sufficient:
| Category | Statement |
|---|---|
| training | tester was not sufficiently trained |
| preparation | rights or requirements were missing |
| functional testing | could have easily been found before |
| Integration | real integration error |
These four categories are clear and simple, and that’s why they work. One team recognized in retrospective that missing rights, for example, had delayed and annoyed the tester, and resolved to do a better job of preparation next time. Annoying a tester who doesn’t have time anyway costs trust, which is difficult to repair.
The attraction of evaluating this more finely with AI is great, but inappropriate as long as the basis is missing. Anyone testing at around the maturity level of 2005 should take a few intermediate steps before resorting to AI. Many suggestions from conferences are also aimed at individual development and are difficult to portability to a configurative SAP context.
Control integration testing via process modeling
Who is responsible when large standard systems are integrated? This question is best answered by process modeling. If a program rethinks the processes instead of just migrating the system, the process parts can be cleanly merged and test runs can be derived from them.
Familiar procedures that experts have always done in this way must first be cast in processes. This may seem cumbersome at first, but it creates a thread that brings the participants together. The process can also be used to discuss whether only the happy path should be tested or which variants should be tested.
There are also user story-independent process tests. The end-to-end routes that are already running are tested again and again because the actual impact of a change is never completely certain.
Why user stories look different in an enterprise context
User stories in the enterprise environment often cover a longer span than a pure configuration switch. The mindset is: I change something to achieve a certain effect, so I test this effect.
The counter-example: when the third gender was introduced, the switch was quickly set. Whether it could then also be used correctly when writing letters may not have been tested. It is precisely this chain of effects that experienced Enterprise people tend to focus on. Ideally, this impact perspective is already included in the refining of the user story and becomes part of the acceptance criteria.
How test data management works at SAP
In the classic SAP environment, you make a productive copy and hope that it does not contain any critical data. A newly set up system does not have this problem because there is simply no data. This creates awareness and forces collaboration.
The test is applied at a specific point: the process bubbles mark where which data comes into play for the first time. Business partners from a certain interface, materials from another. This transparency helps everyone to talk to the respective provider in good time when data is needed.
It is important to differentiate between configurative basic data and transaction data. Basic data is developed as part of the program, transaction data is not. In addition, a hard rule often applies to intellectual property: no external tester gets to see this data.
Test automation needs success stories first
Test automation in the enterprise environment works on two levels. Developers automate what is actually developed using their own tools such as Cypress or unit testers. The process level, which spans several tools, is more difficult.
An introduced tool such as Tosca was intended to empower less technical people here, but failed in reality. It remains an expert tool that you have to familiarize yourself with, and users have no free time for it. Without a sense of achievement, it was not accepted.
The new approach turns this around: first build a central automation team that produces the success stories. Once a basis has been established and it becomes clear that automation is reducing the workload, people in the departments can get involved and become familiar with it. Success before breadth.
For overarching processes that affect several tools, a central orientation is required. Writing the first three tests in one tool, the next in another and connecting them via a third tool is not a desirable state.
Metrics that point in the wrong direction but are still useful
Test coverage that counts one test case per story says nothing about its quality. Whether the one test case is good remains open. As a statement about test coverage, the metric is weak.
Nevertheless, it has a value. The red and blue bars on the charts are easy to explain, quickly visible and suitable for management. They can be used to generate attention and get people to spend more time testing.
Those who know the lever use it consciously. A metric that technically points in the wrong direction but gives people more testing time is worth the money as long as it results in better test cases later on.
Where the next big lever lies: exploratory testing
The biggest untapped potential in enterprise testing is the experienced users and their exploratory testing. They have years of experience with the system and are very receptive if they are shown how to hone their intuitive approach.
Experience-based techniques fit particularly well here, such as heuristics in the form of a compact cheat sheet. Users just have to take the short time to learn them. This is where you break down open doors.
Automation and exploratory testing complement each other, but appeal to different target groups. As long as automation is still thin on the ground, good exploratory testing goes a long way, especially in an environment where experienced users are close to the system anyway and have a strong vested interest in making sure it works. After all, they have to live with it later.
