Next-gen software engineering refers to the approach of dovetailing software design and quality assurance more closely: Every software specification corresponds to a test specification, every model corresponds to a test model. AI-supported automation increases the degree of test automation, but does not replace experts. Critical software needs practically trained testers more than ever.
Key Takeaways
- AI-generated code causes high costs in the long term because it is often of lower quality and still has to be versioned, documented and tested.
- Test specifications are de facto test models, even if the industry doesn’t call them that: they keep business logic consistent across technologies, while underlying technologies change.
- The testing role has a bright future because software is being used in increasingly critical areas and quality assurance cannot be fully automated.
- Continuing education in software testing must focus more on hands-on experience because only action-based learning builds real expertise that theory alone cannot provide.
Testing can only determine whether software is good or bad
Testing has a systematic limit: at the end of a test run, the only statement is whether something works or not. This limit is the starting point for a rethink in software engineering.
Ina Schieferdecker has worked on the quality and testing side for decades, consistently automating the process. Test automation has been widely adopted and the generic test architecture from the ISTQB Test Automation Engineer area is now being applied to frameworks such as the Robot Framework. Nevertheless, there is still a frustration: “Making a statement is not enough.
The conclusion is meant to be constructive. Software design and software analysis must be more closely linked instead of checking what development has produced downstream.
Why software and test software should be developed against each other
Every software system has a test system, every software specification has a test specification, every software model has a test model. This principle can be explained right down to the requirements.
The V-model, which is widely used in German-speaking countries, serves as a reference. Its document-based approach covers not only the software side, but also the test side, and aligns the two. Software and test software are thus developed in parallel and against each other.
The idea extends to the requirements themselves. These should be defined from the system perspective and from the test perspective. If requirements are not testable, development is not worthwhile.
Model-based testing takes to the streets under a new name
Low-code and no-code approaches are bringing model-based testing into practice: the new term is accepted where the old one was difficult to communicate.
Many presentations today speak of test specifications formulated in structured testing. From a mathematical point of view, these are models, even if they are not called that. They are test models.
The advantage of the specification level can be seen in comparison to programming. Anyone who programs keyword-driven code or maps the page logic of a web interface has to delve deep into the details, and such code is difficult to maintain. At specification level, the logic can be formulated across all technologies. The technology can evolve, the business logic is passed.
Vibe coding gets an expensive problem through the back door
Bad code that is waved through quickly becomes very expensive in the long run. This is precisely why interception belongs on the design side, i.e. up front, in the requirements.
This is a shift left approach under a different name. Testing starts with requirements engineering. If the requirements are not testable, there is no chance of building a usable system.
Automatically generated code comes with its own risks. Not only can it be functionally incorrect, but it can also be poor in terms of performance or be created as spaghetti code. Language models have not yet been trained to produce exclusive, very good code, but pull everything in.
We are creating a major problem via the back door, because poorer code that is waved through too quickly becomes very, very expensive in the long run. Ina Schieferdecker
Every additional line of code costs
The wow effect of AI generation masks the follow-up costs. Generated code has to be appraised, updated, versioned, documented and tested. So far, hardly anyone has done this math.
Language models produce a lot of text and code that cries out for appraisal. Even the request to be brief and concise changes little: the result remains eloquent and presents itself.
Ina reckons that a serious productive breakthrough will take another five to ten years. Those who seriously use LLMs or smaller language models are already on the road to disillusionment and understand better what is not possible.
How the ModelBus embeds AI in software engineering
Software engineering is information processing engineering, and all information can be modeled. Requirements need different models than tests, tests need different models than system designs or architectures.
The ModelBus concept developed at Fraunhofer follows the idea of a service bus, but links models and information. Information flows via the bus from the requirement to the system design or test design, from the test execution back to the system analysis and into the readjustment of the requirements.
There are frontrunner tools for every engineering task, but never a one-size-fits-all solution. If AI is used at every point, the question of how the individual chatbots learn from each other what is currently happening in which task becomes interesting. There is not yet a complete picture of this tool class.
AI makes bottom-up models possible
Traditional model-driven software engineering works top-down: you design in models, but at some point the practical side of development catches up, the models are no longer adjusted and fall out of sync.
AI brings a new element. Patterns can be recognized from code, and patterns are abstractions, i.e. models. This makes it possible not only to model top-down, but also to draw bottom-up models from existing code.
Software is language, more formalized than natural language, and therefore a good field of application for LLMs. The same applies to models. This is open research terrain where many doctorates can start.
Theory is not enough for the next generation of testers
Training in the testing profession today is heavily theory-based. There is a glossary and a living syllabus framework with foundation, advanced and expert levels as well as specialists, but practical exercises are often not part of the exam.
This is becoming a problem because software is being used in increasingly critical areas. In critical infrastructures and business-critical applications, software must function reliably, securely and with high performance. Experts ensure this quality, and their skills do not come from theory alone.
There is also a concern: language models draw in broad knowledge while there is little time to train the next generation of experts. Further training should therefore consistently push participants into challenging practical tasks so that real aha and learning effects are created.
Practical exercises create the transfer that abstract examples cannot
Learning effects arise from real tasks, not from abstract school examples. An ATM or an isolated input field carries through the test, but the transfer only succeeds when participants bring their own requirements and user stories and develop test cases based on them.
Ina has taught the Foundation Level at several universities, with a continuous series of practical exercises over a semester or a two-week block course. A not-too-simple use case was worked through: at unit, integration, system and acceptance level, with performance questions and model-based, pulled down to the current tools.
AI lowers the hurdle for precisely this approach. The effort involved in programming and maintaining exercises has long been an obstacle. With AI, this is easier to manage and there is hardly any excuse to leave practice out of training.
The tester role has a bright future
The more critical the areas in which software runs become, the more indispensable quality assurance becomes, and this cannot be rationalized away by automation.
AI takes the level of automation one step higher, and that’s a good thing. The benefit lies in using limited testing resources more intelligently, targeting pain points as early as possible. This is exactly what the profession should be working towards.
