Critical thinking is becoming increasingly crucial for software testers as AI tools reshape the way quality assurance is performed. While automation and machine learning offer new possibilities for collaboration and time savings, they also introduce challenges around bias, validation, and over-reliance. Testers are urged to repurpose their analytical skills—to scrutinize AI-generated outputs, question sources, and avoid complacency.
In this episode, I talk with Steve Watson about critical thinking in the age of AI in testing. Steve says treat AI like a smart teammate. Useful, but you still check its work. We talk bias, missing context, and why lazy shortcuts tempt us. He shares where AI helped, like clustering survey responses, and where it missed ambiguities in requirements. We look at our craft: Ask better questions. Focus on the user. Let tools draft, but you decide. Train the next generation in skepticism and analysis. Same mission. New habits.
"I think we need to use the skills and knowledge that we have and the skepticism that we ought to have and make sure that we pass that on." - Steve Watson
Steve Watson is an experienced Quality Engineering Manager who leads and coaches testing teams to help them to be the best testers they can be.
As a Senior Quality Engineering Manager at easyJet, he is responsible for the overall testing strategy and standards within the Airline Operations area, managing an offshore testing function and the testing budget. His mission is to ensure that Quality Engineering is embedded as early as possible to add value, working with key business and IT stakeholders to identify improvement areas and gain buy-in and support to implement change.
Steve has vast experience in a variety of domains - banking & finance, vehicle leasing, chemical pricing and aerospace, and other than a brief stint as a Product Manager and then Project Manager in 2018, he has spent most of his professional career in the testing & quality engineering space.
As a member of the British Computer Society, Steve cares about improving how we test as an industry, and he writes a blog to share his personal thoughts and ideas.
Steve has spoken at a number of conferences over the past 12 years, including the National Software Testing Conference, Test Expo, UKStar and HUSTEF, facilitated a Zoom roundtable event, ran a BDD interactive workshop at a UK Meetup, and has been a guest on the Testing Peers podcast.
When Steve is not testing, you’ll find him broadcasting a Saturday morning radio show on a local community station in East Sussex, UK, where he has volunteered for the past 12 years.
Sometimes the best ideas are born not in meeting rooms, but in the lively halls of a tech conference. That’s exactly how this episode of Software Testing Unleashed opened: Richie, podcast host and software quality coach, welcomed listeners from the Gustev Conference 2025 in Budapest. The conversation got rolling with a focus on “critical thinking”—a skill whose importance is growing alongside the rise of AI, especially in software testing.
Steve Watson, quality engineering manager and guest on the show, laid out a key distinction. In the past, test automation tools simply did what we told them—they were programmed step by step. Now, AI tools offer suggestions, write code samples, and summarize information. As Steve Watson put it, we’ve moved from “telling a tool what we wanted to do” to “collaborating with a tool.” This shift, while exciting, demands a change in mindset for testers. No longer are we just the operators; we must also be wise collaborators, understanding not only what an AI produces, but also how and why.
Why is critical thinking so essential now? For Steve Watson, it comes down to the issue of trust and bias. AI tools are only as good as the data they’re trained on. Their recommendations can carry hidden biases, based on sources that may be incomplete or skewed. “If you go to any AI tool and you ask a question… The information they provide is only as good as the information sources that they're using,” Steve Watson warned. That’s why testers must question the origin of AI-generated outputs, check for quality, and never accept answers blindly.
Critical thinking isn’t just about finding bugs, either. It’s about uncovering information so decisions can be made—all the more important when the information itself may have been filtered through potentially biased models.
Is this new world forcing testers to relearn everything? Not necessarily. As Steve Watson explained, testers have always reviewed requirements, sought ambiguities, and asked tough questions. The challenge now is to apply those same skills to the outputs of AI, asking where reference data comes from and what might be missing.
Humans, as both speakers joked, are prone to shortcuts—“we are all inherently lazy,” Steve Watson said with a smile. AI tools, like ChatGPT and Gemini, make it very tempting to accept smooth, well-written outputs without a second thought. Testers must resist that urge and make sure critical thinking remains front and center, even if the path to answers looks deceptively easy.
AI offers real benefits, particularly for time-consuming, data-heavy analysis. Steve Watson shared a story of using AI to process responses from user research, finding themes, differences, and presenting results in a fraction of the time manual analysis would take. But he also cautioned that when evaluating ambiguous requirements, AI could miss important edge cases. The lesson: learn what AI does well, let it save time where possible, but know when human judgment is indispensable.
One theme was especially clear: testers must help newcomers build critical thinking skills. Test trainings and workshops often focus on automation and coding, but asking smart questions and challenging assumptions is what keeps software quality high. “Steve Watson” expressed plans to encourage more dedicated education focused on questioning and analysis—skills that will only grow in importance as “AI does half your job for you.”
Is the tester’s job changing? The answer is yes—and no. The core function remains: ensuring software works for end users, no matter who (or what) wrote the code. But tools will continue to evolve, and the knowledge required will shift. As AI takes over more routine work, testers can shift energy to analysis, validation, and advocating for better requirements—keeping themselves relevant no matter what the future brings.