Tools Don’t Solve Test Automation

Test automation strategy is about defining the problem you want to solve before selecting any tool. It starts with mapping current and target states across all test levels, typically visualized as a test pyramid covering unit, API, and UI testing. That gap defines the scope of work. Tooling, framework design, and automation architecture follow only after the problem is clear.

Key Takeaways

• Tool selection only makes sense after the team has defined which problem test automation is meant to solve and what value it should return.
• A test automation pyramid visualized as current state versus target state gives non-technical stakeholders a concrete picture of where effort and money should shift.
• Testing the same functionality at multiple pyramid levels is not redundant: unit tests assess code quality, while UI tests verify that integrated user journeys work as intended.
• The test automation framework (scripts, business logic layer, code libraries) is only one part of the test automation solution; pipeline integration and external system connections are equally part of it.
• Without at least one person on the team who already has test automation experience, an adoption effort will either fail or cost far more than planned.

Why test automation needs a problem before it needs a tool

Test automation starts with a question that is often skipped: what problem are you actually solving? Buying a tool because it looks new or because a salesperson made a convincing pitch leaves teams with software they do not know how to use and no plan for what it should achieve.

Péter Földházi, who has spent more than ten years in test automation and co-authored the ISTQB syllabi on test automation engineering and strategy, works tool-agnostic. The tool comes after the diagnosis, not before it.

The first thing to nail down is scope and approach. Where does automation integrate into the wider ecosystem, what kind of testing does it cover, and at which stages does it run? Once those answers exist, the tooling decision follows almost naturally.

There is no general winner among tools. For an Angular project, Cypress might fit. For a team that has used Selenium for years, switching to Cypress or Playwright rarely makes sense, because the existing knowledge has value. Match the tool to the context, not to the trend.

Manual testing did not die, and that tells you something

More than a decade ago, the common claim was that manual testing was finished and everything would be automated. That prediction did not hold.

More companies do test automation now, including many that previously only tested manually. That is genuine progress. But automation is not a default everyone owes the industry. It is a solution that only earns its place when it answers a real challenge.

Done without a clear purpose, or staffed with too many people, automation can cost more than it returns. The question to keep asking is what value the automation brings against what it costs.

What goals does test automation actually serve?

Test automation serves the goals you can name, and the clearest way to name them is to look at where defects come from.

Consider a real assessment of a gaming company. The starting point was not their coding practices or existing automation, but the defect picture overall. Over an eight-week period, they found almost 5,000 defects, and they were able to resolve only about 30 percent of them.

The consequence was visible in their delivery. Every few months they had to spend two or three sprints on defect fixing alone, which dropped velocity to zero during those windows.

That is the kind of problem automation can attack. With better QA practices and automation that catches issues earlier in the life cycle, the resolution rate might climb toward 50 percent, and the reported defect count might fall from 5,000 to 2,000 because problems surface sooner. The value is concrete, not abstract.

The test pyramid makes the current state visible to everyone

Before building a strategy, run an assessment and map the current state onto the test automation pyramid. This shows which kinds of testing exist and how much automation sits on each level.

The pyramid works because it does not require technical fluency to read. A C-suite stakeholder who controls the budget but does not know what an API contract is can still see the picture: heavy focus on the UI level, defects still found late, money spent late on testing.

From that picture, the case for shift-left testing becomes easy to make. You can show where testing could move earlier, and you can sketch where the team could realistically be in six months. That visual is what turns an abstract argument into a decision.

The ideal pyramid shape is usually not the actual goal. The practical target is what the team can achieve within six months, not a textbook silhouette.

How do you turn the pyramid into a strategy?

The gap between the current state and the future state on the pyramid is your scope of work. That delta defines the strategy.

Say the delta points to more functional API testing plus the introduction of contract testing. Those two types of testing become the focus of the strategy, and automating them becomes the plan.

From there, the questions get concrete. Do you already have API testing? Is it already automated? Do you need to build a framework at all, or can you adopt one that is already well built, so you avoid spending weeks on construction?

Write the plan down, then test it with a proof of concept. A tool evaluation period tells you whether a tool delivers what you need before you commit to it for everything that follows.

You also reassess the existing test cases. If you are adding many API tests, you may no longer need some of the UI tests. The delta drives those trade-offs too.

The same feature on two levels is not duplicate testing

Testing a feature at the unit level and again at the UI level is not redundancy, because the two levels test for different things.

Unit testing is about the quality of the code. UI testing is about whether everything works as intended once it is all integrated, seen from the outside.

Users do not move through the code. They move through user journeys. UI-level tests follow those journeys, which is a different kind of testing value than checking a unit in isolation.

This is why you work across levels with developers and other testers. You need to know what already exists on each level, and you need to know that coverage on one level does not automatically cover the concern of another.

The four ISTQB layers, explained plainly

The ISTQB syllabi describe four layers that often confuse practitioners. People have their Selenium or their Cypress and wonder what the rest even means. The layers move from generic to concrete.

The GTAA, the generic test automation architecture, describes what any framework needs in general to work and to integrate into a solution. It is not a concrete design. It lists the generic capabilities an architecture should have.

The TAA, the test automation architecture, is a concrete design for a specific context. Your TAA differs from someone else’s because you work on a different project, and a single company can hold several TAAs at once.

The test automation solution is the implementation of the TAA, where the TAA acts as the blueprint. The solution is broader than the framework. It also covers integration into pipelines and into management systems, and increasingly the use of MCP to bring AI agents into an editor for generating tests with relevant data.

The framework, the TAF, sits inside the solution as a smaller part. It has three layers: the test scripts, the business logic layer, and the code libraries.

Where AI genuinely changes test automation

The line that matters is between a tool that mentions AI capabilities and a tool that delivers real agentic value. Naming is cheap; agentic work is the differentiator.

Prompting has moved past simple chat toward agents. An agent can take on tasks that are time-consuming or unrewarding and generate them for you, which frees you to focus on work you find more engaging. That shift is already underway and looks set to grow.

MCP is the other practical change. The ability to bring enterprise data into AI existed before, through RAG, but MCP makes it accessible to more teams, and having it inside the development environment matters. You can stay in your editor, in a tool like Cursor or Windsurf, and still get generation from the agents without opening extra tabs.

Will AI take over building the automation?

Not fully, at least not yet. When you generate something with an AI model, you still need to make changes to it.

That has two consequences worth holding onto. First, you have to learn to configure your agents better and to feed them the right data, while ensuring that the relevant data actually gets used. Second, the human in the loop stays important.

What may shift is where that human attention lands. If an agent generates the test cases that a tester would have covered, the developer can take on the test automation side. The center of gravity moves toward coding. A clearer view of how this plays out is maybe half a year to a year away.

One mindset helps here, and it came from a university professor: do not prepare only for the current trend and current best practices, try to figure out what the next thing will be.

We shouldn’t be preparing for what is the current trend and what are the current best practices. We should try to figure out what’s gonna be the next thing. — Péter Földházi

How should a manual-only team start with automation?

Start with staffing, not tooling. The first question is whether anyone on the team already has relevant test automation experience.

If no one does, a developer with a coding background can begin, but the learning curve is real and has to be budgeted as time. The alternative is to bring in a test automation engineer or external services to kickstart the effort.

Skip this step and the effort tends to fail or run too expensive, because the relevant expertise is not there to guide it. Get the staffing right first.

Then return to the same question that governs everything else: what challenges do you want to address, and what are the benefits of addressing them? Strategy and architecture come before the tool, and the problem comes before all of it.