AI is changing software testing at its core. The tester’s job isn’t going away, but it is moving from executing test cases toward test strategy, critical questioning and quality assurance. Flaky tests undermine trust in pipelines and should be isolated and analyzed. The easiest route to shift left runs through refinements, where a shared understanding forms early.
Key Takeaways
- Flaky tests destroy trust in pipelines: if you leave them in, red builds start getting ignored and real bugs slip through.
- Test automation close to 100 percent only works when the software architecture is designed for it from the start, not bolted onto any project after the fact.
- The most direct way into shift left is through planning and refinements, because that’s where testers can help shape a shared understanding of the requirements early.
- Switching from Selenium to Playwright isn’t justified just because the tool is newer, only when concrete problems such as a lack of community or poor integration call for it.
- Manual testers don’t necessarily need to learn programming, because the bigger lever lies in test strategies, test data and using AI support deliberately.
Will AI Replace Software Testers?
Will AI replace software testers? No. The tester role isn’t disappearing, but the job is changing a lot right now. Ask the same question two years ago and you might have got a different answer. Today, more speaks against AI replacing the profession than for it.
AI is changing how software gets built. Teams are experimenting in every direction; some things work, some don’t. Many organizations are in exactly this phase of figuring things out, and there’s a lot of friction along the way.
Some activities will move to the machine, that much is clear. But critical questioning, tracking down problems, building communication and developing test strategies stay human work. If anything, there will be more of it, not less.
Many development projects are hitting the gas on writing code. The quality of what AI churns out comes with a question mark, and someone has to check it. That gives testing more weight again as the way to build trust in software, and building that trust is the core job of testers.
Manual Testers Don’t Have to Learn Programming
If you test manually, you don’t necessarily have to get into programming. You can, but you don’t have to. Programming isn’t the skill that decides your future.
The more valuable question is how quality can improve and what the processes look like. A technical understanding of how software development works helps, because it lets testers contribute in more places. Learning hardcore programming isn’t part of that.
Being open to it doesn’t hurt, though. A bit of scripting is a good way in, and today you can use AI to learn, not only to generate code. It can help you build programming knowledge instead of just accepting finished results.
The bigger levers are elsewhere: test strategies, test data and working with AI itself. Knowing how AI can support testers in their work is a skill that’s needed more and more.
Why Switching Tools Isn’t an End in Itself
Changing your automation tool only pays off if it solves a specific problem. Being modern isn’t a reason on its own. The right question is: what do I actually want to fix by switching?
Selenium has been around for many years, has a large community and often runs very reliably. Some people no longer consider it fancy, which is why some teams are moving from Selenium or Cypress to Playwright simply because it’s cool and fashionable.
This is where a core tester skill comes in: critical questioning. If the current solution works well, nobody is unhappy with it and it will carry you through the next few years, you can stay where you are, even if the tool isn’t the trendiest one.
There are good reasons to switch: more people around you are working with the new tool, or it can be hooked in closer to development. You need to answer those questions up front. There’s no blanket answer.
The other side is just as real. At some point, tools become outdated and end up in the archive. If you miss every train, you end up alone on an empty platform. Selenium hasn’t reached that point yet.
One Hundred Percent Automation Is Mostly Nonsense
The goal of automating everything fails in the vast majority of projects. It depends, and in reality the circumstances often speak against it.
A common misunderstanding starts with the term itself. Test automation is often equated with user interfaces, business processes and end-to-end tests. But unit tests are automated tests too, because nobody runs them by hand. You can automate across all test levels, and a high degree of automation is possible.
Some projects automate close to one hundred percent and have good experiences with it. Everything fits together there: the software architecture is designed for it, and so are the domain setup, the delivery processes and the tools. Manual checks are then just a quick look. Much of it comes down to the application and how it is structured.
In most projects, something gets in the way: bad architecture decisions from years past, an unsuitable tech stack, the wrong tools, missing knowledge or developers who won’t play along. Then people start tinkering and try to beat the problem with the latest tool. It makes more sense to ask where automation really carries its weight and to work outward from there.
Isolate Flaky Tests First, Then Analyze Them
The biggest danger with flaky tests is lost trust. If a test in the pipeline is sometimes red and sometimes not, and nobody knows why, people stop trusting the pipeline and the tests.
The first step is to take flaky tests out of the pipeline right away. The goal is a pipeline that is consistently stable. If something turns red after that, it really is a problem.
The second step is analysis. Where does the instability come from? There are many reasons a test may not run reliably. Some can be fixed with more or less effort, and then you decide whether it’s worth it.
Sometimes you delete a flaky test and replace it with two new ones, or cover the case manually because there’s no other technical option. That points back to the architecture: what can’t be automated reliably was often never designed for it. That doesn’t have to be a flaw if it was a conscious decision at the start of the project.
Shift Left Starts in Refinement, Not in Code Review
The best way into shift left is through planning and refinement, not through code review. Testers don’t need to dive straight into the code with developers or explain to them how to write better unit tests.
In refinements, testers work as equals with the rest of the team on shaping requirements and user stories. That’s where you learn early what is going to be built. You can ask questions and help form a shared understanding that includes your contribution to quality.
It isn’t always easy. It often starts as a status issue, because refinement is seen as the development team’s business, and the tester only gets to estimate the testing effort at the end. That misses the point. Once you’ve built up some trust, you can make a real difference here.
At its core, agile work is always about shared understanding. Whether through Specification by Example, BDD or another method, the question stays the same: what do we want to build? Everyone should share that understanding, whether they come from business, engineering or testing, so the implementation succeeds.
Test Strategy Belongs Back on the Table
The AI wave has brought many teams back to an old question: what are we actually doing with testing? Companies are digging out old test plans containing a test strategy nobody quite remembers.
Often what’s written there no longer matches reality. Testing is done the way it has always been done. Many teams are about to take a hard look at that again.
This is about the risk-based approach and prioritization. Which quality characteristics get tested, how deeply and why? Above all: what is the goal behind testing in this project or product?
These questions belong on the table for the whole team, with the simple follow-up question of whether everything still fits. This is exactly where new needs are emerging right now, driven by the changes AI brings to development.
It Pays to Borrow from Other Disciplines
Topics that seem to have nothing to do with testing at first glance feed straight into quality work. Conflict management and soft skills carry over into everyday testing and raise the question of what they mean in concrete terms for the quality business.
The same goes for the technical side. Enterprise testing, testing microservices or modernizing legacy systems with AI are fields testers don’t dig into every day, and there is a lot to take away from them.
The learning works in both directions. If you regularly look for new perspectives, you keep finding ideas that enrich your own day-to-day testing. That’s another reason to look beyond pure test technique.
Frequently Asked Questions
Which testing tasks will remain the domain of humans as AI generates more and more code and tests?
Critical analysis, identifying problems, establishing communication, and developing test strategies remain human tasks. Individual activities will be taken over by machines, that much is clear. Because many projects are rushing through code development and the quality of AI results remains uncertain, someone has to verify them. As a result, testing is regaining importance as a means of building trust in software.
What skills are more valuable to testers than programming knowledge?
Test strategies, test data, and working with AI itself are more valuable in the long run. A technical understanding of how software development works is helpful because it allows testers to make an impact in more areas. Learning hardcore programming isn’t part of that. If you’re interested, start with some scripting and use AI to learn, rather than just accepting ready-made results.
When does switching test automation tools justify the effort?
Only if it solves a specific problem. Being modern isn’t a reason on its own. Good reasons include more people in your team working with the new tool or the fact that it can be integrated more closely with development. If the existing solution is stable, no one is dissatisfied, and it will serve its purpose for the next few years, you’re free to stick with it. However, if you miss every opportunity, you’ll be left standing on a lonely track.
Do unit tests actually count as test automation?
Yes. No one performs them manually, so they’re automated tests. Test automation is often equated with user interfaces, business processes, and end-to-end testing, and that’s precisely where the misunderstanding lies. Automation can be applied across all test levels, which is how a high degree of automation becomes achievable in the first place.
Why don’t most projects achieve full test automation?
Because something isn’t working: poor architectural decisions from years past, an unsuitable tech stack, the wrong tools, a lack of knowledge, or developers who aren’t on board. Projects that come close to 100 percent have aligned their software architecture, domain expertise, implementation processes, and tools. Instead of trying to force a solution with the latest tool, it’s worth asking where automation really makes a difference.
What should you do with tests that sometimes show red and sometimes green in the pipeline?
First, isolate them; then analyze them. Unstable tests are immediately removed from the pipeline so that it remains consistently stable and a red result once again indicates a real problem. Next, you identify the cause and decide whether fixing it is worth the effort. Sometimes a test is deleted and replaced with two new ones, or the case is covered manually.
Why is it difficult for testers to truly influence the refinement process?
It often starts as a status issue: Refinement is seen as the development team’s responsibility, and the tester is only allowed to provide an estimate of their testing effort at the end. What’s actually meant is something else: collaborating on an equal footing to shape requirements and user stories, asking questions, and contributing to quality. Those who have earned trust can make a difference there.
How can you tell that a test strategy needs to be revised?
A clear sign is a test plan whose strategy no one knows exactly anymore, while testing continues as usual. In that case, the document no longer reflects reality. A risk-based approach and prioritization need to be brought to the table: Which quality characteristics are being tested, to what depth, why, and what is the goal behind the testing in this project or product?
