Software testing is changing fundamentally as a result of AI: the job of a tester is not becoming redundant, but is shifting away from pure test case execution towards test strategy, critical questioning and quality assurance. Flaky tests jeopardize trust in pipelines and should be isolated and analyzed. The easiest way to achieve shift left is through refinements, where a common understanding is established early on.
Key Takeaways
- Flaky tests destroy confidence in pipelines: if you leave them running, you risk red builds being ignored and real bugs going undetected.
- Test automation close to 100 percent only works if the software architecture is designed for it from the outset, not as an afterthought in arbitrary projects.
- The most direct way to get started with shift left is through planning and refinements, because testers can help shape a common understanding of requirements at an early stage.
- A tool change from Selenium to Playwright is not justified by timeliness alone, but only if concrete problems such as a lack of community or poor integration trigger it.
- Manual testers do not necessarily have to learn programming, because the greater leverage lies in test strategies, test data and the targeted use of AI support.
Will AI make the tester job redundant?
No. The role of the tester is not disappearing, but the job is changing massively. If you had asked two years ago, you might have gotten a different answer. Today, there are more arguments against AI replacing the profession.
AI is changing the way software is developed. Teams are trying things out in all directions, some things work, some don’t. Many organizations are currently in this discovery phase, and they are rubbing up against each other loudly.
Individual activities are moving to the machine, that is foreseeable. But critical questioning, tracking down problems, establishing communication and developing test strategies will remain human work. There will be more of it rather than less.
Many development projects are currently stepping on the gas when it comes to creating code. There are question marks over the quality of what AI spits out. Someone has to check it. Testing is once again becoming more important than building trust in software, and that is the core task of testers.
Manual testers don’t have to learn programming
Manual testers don’t necessarily have to learn programming. You can, but you don’t have to. Programming is not the skill that determines your future.
What is more valuable is the question of how quality can be improved and what the processes look like. A technical understanding of how software development works helps because it allows testers to work in more places. Learning hardcore programming is not one of them.
Nevertheless, openness doesn’t hurt. Starting with some scripting is a good place to start, and AI can also be used for learning today, not just for generating code. You can use it to build up programming knowledge instead of just accepting finished results.
The more important levers lie elsewhere: in test strategies, in test data and in dealing with AI itself. How AI supports testers in their work is a skill that is increasingly needed.
Why a tool change is not an end in itself
A change of automation tool is only worthwhile if it solves a specific problem. Modernity alone is not a reason. The right question is: What do I actually want to solve with the change?
Selenium has been around for many years, has a large community and is often very stable. It is no longer considered fancy by some, which is why some people are switching from Selenium or Cypress to Playwright because it is cool and en vogue.
This demonstrates a core skill of testers: critical scrutiny. If the existing solution is working well, no one is dissatisfied and it will last for the next few years, then you can stay where you are, even if the tool is not the most popular.
There are good reasons for a change: more people in the environment are working with the new tool, or it can be docked closer to the development. You have to answer such questions beforehand. There is no one-size-fits-all answer.
The other side is just as real. At some point, tools become obsolete and end up in the archive. If you miss every train, you end up standing on a lonely track. With Selenium, this point has not yet been reached.
One hundred percent automation is mostly nonsense
The goal of automating everything fails in the vast majority of projects. It depends, and in reality, circumstances often speak against it.
A common misunderstanding is already inherent in the term. Test automation is often equated with interfaces, business processes and end-to-end. However, unit tests are also automated tests, because nobody performs them manually. Automation is possible across all test levels, and a high degree of automation is possible.
There are projects that automate close to one hundred percent and have good experiences with it. Everything fits together there: The software architecture is aligned with this, as are the technicalities, the implementation processes and the tools. Manual checks are then only carried out briefly. This mainly depends on the application and its structure.
Something doesn’t work in most projects. Bad architectural decisions from previous years, an unsuitable tech stack, the wrong tools, a lack of knowledge or developers who don’t play along. Then people fiddle around and try to kill something with the latest tool. It makes more sense to ask where automation really makes sense and to work from there.
Isolate flaky tests first, then analyze them
The biggest danger with flaky tests is the loss of trust. If a test in the pipeline is sometimes red and sometimes not, and nobody knows why, trust in the pipeline and in the tests is lost.
The first step is to immediately isolate flaky tests from the pipeline. The goal is a consistently stable pipeline. If something is then red, it really is a problem.
The second step is to analyze it. Where does the instability come from? There are many reasons why a test is not stable. Some can be fixed with more or less effort, and then you decide whether it’s worth it.
Sometimes you delete a flaky test and replace it with two new ones or cover the case manually because there’s no other way technically. This points back to the architecture: what cannot be automated in a stable way was often never designed for this. This doesn’t have to be a flaw if it was consciously decided at the start of the project.
Shift left starts in refinement, not in code review
The best way to get started with shift left is through planning and refinement, not code review. Testers don’t need to get right into the code with developers or explain to them how to do unit tests better.
In refinements, testers work at eye level with the other team members on the design of requirements and user stories. This allows them to understand early on what is to be implemented. You can ask questions and help shape a shared understanding that incorporates your contribution to quality.
It’s not always easy. It often starts as a status issue because refinement is considered to be the responsibility of the development team and the tester is allowed to estimate their testing effort at the end. That’s not the point. If you have built up some trust, you can really make an impact here.
At its core, agile working is always about a shared understanding. Whether via Examples, BDD or another method, the question remains the same: what do we want to implement? This understanding should be the same for everyone, whether from business, technology or testing, so that implementation is successful.
Test strategy belongs back on the table
The AI wave has brought many teams back to the old question of what they are actually doing with testing. Companies are digging out old test plans that contain a test strategy that no one remembers exactly.
Often, what is written there no longer corresponds to reality. Testing is done the way it has always been done. Many people are about to take a critical look at this again.
This concerns the risk-based approach and prioritization. Which quality characteristics are tested, to what depth and why? Above all: what is the goal behind testing in this project or product?
These questions need to be on the table for the whole team, with the simple follow-up question of whether everything still fits. This is exactly where new needs are emerging, driven by the changes that AI is bringing to development.
Transfer from other disciplines is worthwhile
Topics that at first glance have nothing to do with testing have a direct impact on quality work. Conflict management and soft skills can be transferred to everyday testing and answer the question of what they mean in concrete terms for the quality business.
The same applies to the technical spectrum. Enterprise testing, the testing of microservices or the modernization of legacy systems with AI are fields that testers are not immersed in on a daily basis and from which a lot can be learned.
The learning effect goes both ways. If you regularly look for new perspectives, you will always find ideas that enrich your own day-to-day work as a tester. It is also worth looking beyond the boundaries of pure testing technology.
