What moves the software testing community?

AI is changing software testing on two levels: as a tool for test data generation, document analysis and code reviews, and as a threat to existing test roles. Deep system knowledge and domain expertise remain indispensable, because AI agents can cover superficial tests, but cannot yet reliably recognize side effects in grown code bases.

Key Takeaways

• AI-supported code generation generates short-term productivity gains, but will trigger a wave of major refactorings and software failures in the medium term because the overall view of grown systems is missing.
• QA lead positions and experienced testing roles are currently being actively rationalized, while regulated industries have so far been much less affected by these job cuts.
• Non-functional requirements such as security, accessibility and performance are considered standard in testing teams, but are often completely unknown to product management and C-level decision-makers.
• System architecture knowledge and domain expertise are the skills that make testers irreplaceable, because an AI can provide high-level coverage but has no contextual knowledge of its own product domain.

AI is changing testing, but not in the way management would like

Artificial intelligence dominates the discussion in the testing community, from conference stages to the local advertising paper. When it comes to testing itself, the topics mainly revolve around generative AI: generating test data, creating test cases, summarizing and evaluating documents.

The next step is agents and the Model Context Protocol (MCP). This allows an agent capability to be set up from an API. Anyone who is testing should get to grips with this now, even without pressure from the company strategy. The tools are freely accessible and you can build and test your own systems.

AI is mature for productive use where a lot of work is done with text and documents. Daniel Knott cites the analysis, summarization and evaluation of documents from his work at Techniker Krankenkasse as a case where AI is already being used in productive applications. Many other companies, on the other hand, are still in the discovery phase and are integrating the first prototypes into existing products.

Test data and new views of the system: this is where AI pays off for testers

Test data generation is one of the strongest AI application areas in testing, especially when access to proprietary data sources is secured via private LLMs. This brings concrete added value to day-to-day work.

A second effect is the changed view of data. Allowing a co-pilot or another LLM to access your own repositories provides new insights into data correlations. This leads to better decisions: Do I want to test at this level, or do I want to automate completely?

The role of the tester as a domain expert does not disappear as a result. On the contrary. Only those who know the technical contexts can evaluate the results of the AI. AI provides more information about a system, the classification remains human work.

A wave of generated code is rolling in, and it will be expensive

Vibe coding and AI-supported development drastically lower the barrier to building software. This is a favorable phase for young founders: with few staff and little technical know-how, a product can be launched more quickly.

The bill comes later. Over the next few years, we can expect a wave of major refactorings, failures and software breakdowns. A “fix” is quickly generated, a code review quickly delegated to AI. What is missing is the overall view: whether a changed line in a grown application triggers a side effect.

This is precisely where new needs arise. People are needed who can understand, penetrate and evaluate entire system landscapes. Daniel assumes that this demand will return as soon as companies have understood what it means to use AI sensibly instead of just using it as a means of saving personnel.

Unfortunately, we now have to get through this valley of redundancies. Very good people in testing and engineering are looking for new jobs on LinkedIn. I believe that this will come back because companies first have to understand what it means to use AI sensibly.

Daniel Knott

Worrying about your own job is real

In addition to AI, the community is primarily concerned about uncertainty about their own future. If you look at LinkedIn every day, you will see search queries and shared applications. AI is currently having a noticeable impact on jobs in testing.

There is a particular lack of experienced positions. QA lead positions and roles in testing management or general management are currently hard to find. Tasks like these are being rationalized away or assigned to someone else.

The pressure is not just coming from AI. The economic situation and an old management mindset are working together. There is often a lack of in-depth insight at C-level, and then the red pencil is quickly applied to testing. Communicating the added value of testing to the business is an old task for the community, but one that is rarely in the hands of the individual at such times.

There are regional differences. In regulated environments, the impact has so far been less than in SaaS applications and start-ups. German-speaking countries lack the hire-and-fire mentality that is more pronounced in the USA. This is no guarantee for the future.

Low-code and no-code remain because test knowledge is lacking

The trend towards low-code and no-code automation tools continues. Companies are resorting to them because they lack testing expertise. Instead of an independent role as a test automation engineer, the tool fills the gap.

At the same time, teams are looking more closely at the levels at which they want to automate. Products have become more complex: lots of backend code, large interfaces, multiple frontends, some desktop and embedded.

As a result, the automation strategy needs to be restructured. UI-driven tests with Cypress or Selenium are important, but expensive to execute. Established test suites are being refactored to a greater extent in order to restructure the strategy along the classic pyramid and extract the added value.

Test pyramid and agile testing quadrants open up communication

In many companies, there is simply a lack of communication between those involved. Developers do their unit and small integration testing, a test department somewhere does surface automation, often for things that would have long been covered at lower levels.

Two models help to close this gap. The test pyramid is suitable for talking to developers about the levels at which code is automated. The agile testing quadrants bring non-functional requirements, acceptance criteria, performance and observability into the picture and thus provide a holistic view of quality.

Daniel uses both models in software testing training for a product boot camp, in front of participants without a testing background. The feedback from the companies: The models help above all because they start the communication in the first place. If that succeeds, a lot is gained.

Quality belongs in the team, not in a separate department

Those who test should be deeply rooted in the product team and the product life cycle. This is the only way to build up the technical and domain expertise over time that is necessary to penetrate the depths of a system.

This has consequences for the skills question. Broad, superficial high-level testing can cover an AI in the future. Agent tools are not yet perfect here, but they already cover simple high-level topics well. This is not a loss, but a relief for the team.

The exciting tension remains: More holistic responsibility in the team makes everyone broader, but no one deeper. This is precisely why there is much to be said for consciously keeping testing expertise within the team again instead of distributing it among everyone until it disappears.

What skills beyond AI carry

Blanket answers fall short because everyone is at a different point. For beginners, the following applies: build up the basics via books, blogs and YouTube, and even take a certification to get to know the terminology of testing.

Beyond that, these fields are particularly worthwhile:

• System architecture and system modeling Modern applications are no longer just backend, frontend and an interface. Cloud systems and nested landscapes are part of it. Those who understand them ask more targeted questions as testers.
• **Industry and business knowledge ** It is the basis for evaluating AI results. AIs hallucinate often, a lot and convincingly, sometimes so well that you don’t realize it.
• **Non-functional requirements ** Old hat for testers, often not in product teams. When asked when non-functional requirements were last discussed, empty faces often follow.
• **Security and accessibility ** Even without expert status, it is possible to read in, try out and test new things here. The field is inexhaustible.

The most important skill is the attitude behind it. Don’t remain at foundation level, but stay in a mode of further development and lifelong learning, with moderation. It is easy to become overwhelmed and stressed by the flood of new topics. A certain flow goes further than panic.

The cycles get faster, and new terrain rewards curiosity

The mobile wave was considered fast around 2010: iPhone 2007, first app stores 2009, then the push around 2010 to 2012. AI has come over the industry even faster. Computing power on mobile and desktop devices continues to drive the cycles.

In such phases, there is still no standard. With AI, new quality criteria emerge for which there are neither books nor established procedures. All that remains is trial and error until a standard emerges.

This is precisely the opportunity. Anyone in a company who gets the time to dig into a new research topic has a rare opportunity, comparable to Jugend forscht. Daniel traces his own path back to this: he was able to share the knowledge from his early mobile days, which resulted in a book and YouTube channel. The testing community thrives on this culture of sharing and support, and that is exactly what drives it forward.