In Wolfgang Platz’s view, three developments will shape the future of software testing. Low-code testing produces citizen testers: business experts who test software in an abstract business definition world without ever touching the code. On the organizational side, testing swings from the central TCoE to the agile teams and on to a lean linked TCoE. And with the one-test definition, functional tests become the basis for performance and security tests.
After successfully building up a test consulting firm in the early 2000s, Wolfgang Platz founded Tricentis in 2007. With more than 20 years of technology experience and broad knowledge of software testing, he has shaped Tricentis significantly, both on the product side and in the software testing methodology that goes with it.
Today he is responsible for Tricentis’ strategy and vision, with the goal of making continuous testing for enterprise DevOps a reality.
The Future of Software Testing: Low-Code Testing, Linked TCoE and One-Test Definition
The Hunger for Software
The way I see it, the dominant force is the hunger for software, and it is massive. There are IDC forecasts saying that 40 percent of all code will be written in the next year alone. That shows how extreme this hunger for software is. By comparison, the growth in new software development talent coming out of universities worldwide is around 20 percent. Per year, we want 35 or 40 percent more software, but there are only 25 percent more developers. So the question is how that is supposed to work.
Microsoft predicts that the future of software development will be less about real development and more and more about parameterization and configuration at an abstract level, and that we will see the rise of the citizen developer. Look at Google today and you also see no-code and low-code development with massive growth, which adds immense momentum. What is interesting is what actually gets built with low-code and no-code. Today, the applications developed there are very simple. The moment you want a real system, there is no way around development.
With the real systems, we see enterprise applications all trying to meet this enormous demand for customization by building massive configuration layers that let you put no-code and low-code development on top of their kernel. Many companies are very active here and are trying hard to simplify. They have their own programs to support low-code and no-code development on their base platform. Not least because that is what makes SaaS possible, which is a must given how much custom development there is. So on one side we have enterprise systems pushing the coding threshold higher and higher, which means you can do more and more with no-code and low-code. On top of that, we have today’s low-code and no-code platforms, whose capabilities are good enough to solve simple things without coding.
Low-Code Testing Creates Citizen Testers
In my view, two things will happen, mainly in the back office. That work used to be done in Excel; now it gets put together with low-code and no-code, and these are all very simple processes. Today, the complexity of these in-house solutions is still below the point where testing is needed. Of course a tester will say everything has to be tested. But we know that simple things don’t get tested.
We will see two things, though. On the one hand, complexity will increase, and on the other, the individual solutions will start to connect. Suddenly there is a level of complexity that needs testing. That is the moment when you have citizen developers and citizen testers. At that moment, we face a big demand for no-code automation, which will follow no-code development. A no-code developer isn’t going to start writing code for their automation. So, with a certain delay, the big momentum of citizen development we will see over the next few years will spill over into the testing discipline.
From a tooling perspective, there will therefore be two worlds. There will still be today’s coding world with its various frameworks, where people try their hand at Selenium and Appium, for example. And alongside it there will be a growing, more abstract business definition world, where people do software testing without having to go into the code. That is my view of how skills will develop.
Pendulum Swings: From the TCoE to the Linked TCoE
On the organizational side, we will see a pendulum swing. We already see it across our customer base. Ten years ago, everyone wanted a TCoE (Test Center of Excellence). That has been over for five years at the latest. With some delay, people realized that the TCoE wasn’t working out and that this big pool of manual testers working order by order was irrelevant. There were all sorts of problems, and many customers responded by saying the TCoE had to be dissolved.
Then came the idea of moving everything into development, which would make dedicated software testing unnecessary. But that raised the question of what the actual principle was. In a large organization, it only works for the first three or five teams. Agile development took it over and got great results. Then it was scaled up, and suddenly there was a rude awakening, because the productivity gains came nowhere near the productivity expectations. How could that happen? When you scale, two things happen. You discover that you do need higher test levels after all: integration, system integration and maybe even user acceptance. It is no longer about one isolated application but about a whole network of systems. The question is who does that. The agile teams all think only up to the edge of their own agile scope, and that is in no way a criticism, because that is how they are set up.
So we arrive at the Hegelian thesis, antithesis and synthesis, with something in the middle that we know as a digital or linked TCoE. You still set up a very lean body that sets or recommends some standards for the test approach, test methodology and automation. When it comes to reporting results, though, it has real governance. It is still a central body, but one that works very closely with the teams. That is a recommendation we have made, and it works well for our customers. Even for the higher test levels in its own agile test teams, it brings together the artifacts from the lower levels and aggregates them, so we get a higher-level test. That is the organizational form I see.
The wrong approach would be to hand everything to the agile teams and hope it gets solved there. The fact is, that doesn’t work. Developers are progressive animals, not regressive animals, which is why they don’t like doing unit tests. Push forward. Make things new. All of this shows up in interesting numbers. We ran a survey among Swiss banks, and it shows that they reach a certain code coverage for release one, meaning for the new tests, and that coverage then steadily drops. The reason is that nobody does the maintenance. The coverage you get from unit tests simply evaporates over time.
That means you de facto have to do something that looks inefficient, because you catch the tests at a higher integration level. But you have the huge advantage that developers no longer have to do it. Now you can bring in people who see software testing as a task and not as a necessary evil. They can maintain and keep these tests alive. Microservice architecture helps a lot here, because there is always an API wrapped around the service. That lets you set up a business logic test at the API level, which is much easier to maintain. In our experience, people do write these contract tests. At that level they are willing to write and maintain them.
One-Test Definition
There is a third topic I wanted to talk to you about, and it is one that interests me personally. It is called the one-test definition. What do I mean by that? We all distinguish between functional and non-functional tests. Among the non-functional ones, I want to pick out the two that take the most effort over time: performance and security.
Here is the thesis. An application exists, or has a right to exist, because it offers some functionality no other application has. It solves some problem in a way no other application does. Why would I need it at all if another one does exactly the same thing? I’d argue that nobody writing software for a business builds redundant functionality. Why would you? If you do, it is only because you don’t know it already exists somewhere else. But if I build new functionality, I will keep building new test cases that have never existed in this form, because the functionality is new as well. So the functional test case portfolio is always individual to a certain degree. If you could use two completely identical test case portfolios, the applications would be identical. So why do it? Finding number one: functional test portfolios are always application-specific.
I keep hammering on this because non-functional ones are not necessarily application-specific. Look at security testing today: you don’t want anyone to be able to carry out intrusions or attacks. These attacks are all highly generic, and you don’t want to see them in any application. It doesn’t matter whether it is a web check-in or a banking app. You want that application to be secure. Most of what security means can be described generically.
It is very similar with performance. For performance, we know that 1.3 seconds is the limit. We don’t want to wait longer than that for a website. There is an application-specific parameter for the level of concurrency at which the use cases have to run. 10,000 people sit on one use case at the same time and 5,000 on another. But that isn’t specific to the application’s functionality. It is a parameter set by the environment, because I have 10,000 employees, say. The use case itself isn’t affected by it.
What does that mean? With these thoughts in mind, we asked ourselves why we can’t take the functional navigation through the application as the basis and enrich it with these generic requirements to build a security or performance test. The hard part of a security test isn’t defining the requirements, it is navigating through the application and trying the nasty intrusions along the way. The functional test has already solved that problem. With performance, too, the requirement isn’t the use case as the user experiences it. I only want to make sure it works 10,000 times at once. So I have to feed in different data and set up the environment data accordingly. But the functional navigation through the application, which we have already built, lets the load be generated in the background. So why can’t I reuse a functional definition for the non-functional area and enrich it there with the generic part? That is what I call the one-test definition.
We have now put this to use at Tricentis and implemented it for the first step, performance. You take the functional test portfolio, add the communication track in the background, and then enrich it with all kinds of data combinations so you can get 10,000 users on it at the same time. But the flow is that of the functional application. Updates and maintenance come from the functional application at the push of a button. Initially, you still have to enrich the functional flow with the specifics of this test type, which are generic. Once that is done, updates and maintenance always happen at the push of a button.
That gives you something new. The moment your functional test turns green, you have achieved two things. First, the application is reasonably stable, otherwise it wouldn’t survive the smoke test. Second, the functional script is maintained. I can build on that maintained state and push the update up to the performance and security tests. I run them right away and have at least a smoke performance test and a smoke security test. As long as everything is green, people who aren’t experts can handle that too. When it turns red, you need someone to take a look. That applies to both security and performance. But you can lift the bulk of the work to another level and get continuity in quality assurance from the non-functional side too. That is the idea behind the one-test definition.
What Testers and Test Managers Should Learn: DevOps, Docker, Kubernetes and Test Methodology
In the future, you will need to understand how DevOps works far better than today. As a test manager, you absolutely need to understand how Docker and Kubernetes basically work. How are they used in principle? That wasn’t necessary in the past. Back then, tests were more like one-off events, and that is no longer the case.
Beyond that, I still see a frightening lack of methodological knowledge among test managers across the board. The mistakes of 20 years ago are still being made. As long as we automate everything, all is well. Nobody asks what additional coverage a test like that actually generates. People still don’t grasp how significant the step from an intuitive to a methodical test approach is. What matters is knowing when to do what. When do we need test data design, when test case design? I recommend this basic test methodology to everyone.
Frequently Asked Questions
Why aren’t there enough new software developers to meet the demand for software?
Demand grew faster than the supply of developers. In 2022, the article cited an IDC forecast stating that 40 percent of all code would be written in the following year, while universities worldwide would only produce about 20 to 25 percent more developers. Parameterization and configuration at an abstract level, as well as the phenomenon of the citizen developer, were seen as potential solutions.
Do applications built with low-code or no-code need to be tested?
Initially, usually not, because their complexity falls below the threshold for testing: these are simple back-office processes that used to be handled with Excel. This changes for two reasons. The complexity of these in-house developments increases, and the individual solution components become interconnected. This creates a need for testing, and citizen development is eventually followed, albeit with a delay, by citizen testing.
Will test automation still require programming skills in the future?
Both worlds coexist. In 2022, the article distinguished the classic world of coding (with frameworks like Selenium and Appium) from a growing, more abstracted world of business definitions, where software is tested without delving into the code. Those developing skills should therefore not rely solely on code, but neither should they dismiss it entirely.
Can testing be fully shifted to agile development teams?
No. The approach works for the first three to five teams but breaks down when scaled: Higher test levels, such as integration, system integration, and user acceptance, affect the entire system ecosystem, whereas agile teams think only within the limits of their scope. Instead, a lean, centralized body (a Linked or Digital TCoE) was recommended to set standards, aggregate results, and work closely with the teams.
Why does the code coverage achieved through unit tests decrease over time?
Because no one performs the maintenance. A survey of Swiss banks showed high coverage at the time of the first release, which then gradually declined. Developers drive new features forward and are reluctant to work in a regressive manner. It is therefore more effective to implement tests at a higher level of integration, where they are maintained by people who view testing as a task rather than a necessary evil.
At what level are automated tests best maintained?
At the API level. Microservice architectures provide API wrapping around each service, allowing business logic tests to be built on top of it, tests that are significantly easier to maintain. Based on the experience described in the article, developers actually wrote and maintained such contract tests, unlike traditional unit tests.
Can functional test cases be reused for performance and security tests?
Yes, and that is precisely the idea behind the “one-test” definition. Functional test portfolios are always application-specific because new functionality requires new test cases. Non-functional requirements, on the other hand, are largely generic: no application wants to see attack patterns, and the article cited 1.3 seconds as the maximum wait time. Functional navigation provides the flow, enriched with generic requirements and data constellations.
What should test managers know about the DevOps infrastructure?
At the very least, they should have a basic understanding of how Docker and Kubernetes work and how they’re used. Tests are no longer isolated events. In addition, 2022 saw criticism of a lack of methodological knowledge: the reflex to fully automate without questioning the additional coverage gained, and the misunderstood transition from intuitive to methodical testing involving test data and test case design.
