Cypress, Playwright or WebdriverIO

Front-end test automation today includes three established tools: Cypress, Playwright and Webdriver I/O. Playwright has the edge because it covers the entire test pyramid, from component testing to end-to-end testing, without the technical limitations of Cypress for multi-tab applications and cross-domain testing.

Key Takeaways

• Playwright has overtaken Cypress in one key aspect of end-to-end testing: multi-tab applications and cross-domain tests are not technologically solvable in Cypress because the tool remains trapped in the browser window.
• Component testing in the frontend works like unit testing in the backend: It checks validations, events and network requests of a single UI component in isolation before costly end-to-end testing becomes necessary.
• Playwright uses the Chrome DevTools protocol for error analysis and thus allows direct tracing of network traffic, tracing and test steps, including jumping to the error location in the code.
• If you don’t want to migrate a running Cypress project, you can cover new features with Playwright and run both suites in parallel, because both are based on TypeScript and the syntax is similar.
• Playwright is actively supported by Microsoft, Cypress has a commercial product, while Webdriver I/O lags behind visibly without comparable resources.

Frontend testing needs the whole test pyramid, not just end-to-end

A frontend test strategy covers two levels: End-to-end testing across the entire system and component testing at the level of individual UI building blocks. Both belong together, even if they require different tools and ways of thinking.

An end-to-end testing checks a use case from start to finish and integrates everything: frontend, backend, database. It simulates a real end user. This requires a tool that can automate the browser, because otherwise this process cannot be mapped.

Component testing is the front-end counterpart to classic unit testing. The unit here is not a function in the backend, but a component: a button, an input field or an entire screen. At this level, validations can be checked, triggered events checked and network requests verified long before end-to-end testing becomes necessary.

The purpose behind this is the same as in the backend: to cover all variants, alternatives and error handling as far down the pyramid as possible. The expensive end-to-end testing no longer has to do what component testing does.

How component testing works technically

Component testing runs on a test bed that must be suitable for each frontend technology. A React component requires a React test bed that is provided by the respective framework.

The component is integrated into this test bed, i.e. mounted. The same mechanics are then used as for end-to-end testing. This is precisely why it is primarily the established end-to-end tools that are now also capable of component testing.

The path is obviously only going well in one direction: it is easier to move from the end-to-end level to the component level than vice versa. For a long time, Cypress was the only tool that mapped this complete stack. Playwright followed suit with component testing, later followed by WebdriverIO.

Why Playwright has now overtaken Cypress

Playwright covers the same test pyramid as Cypress, without its technical limitations. This makes it the obvious choice for new projects.

Cypress is limited by its architecture to a single browser window. This makes multi-tab applications and cross-domain testing difficult or even impossible because the tests are effectively trapped in an iframe. Playwright does not have this limitation and maps such scenarios much more easily.

Playwright is also not limited to TypeScript and JavaScript, but supports other languages and is easier to integrate into other testing tools. Cypress remains very close to development, but is narrower in scope.

When it comes to error analysis, Playwright continues what Cypress made famous with its flipbook: tracking what has happened in the application over time. Playwright uses the Chrome DevTools protocol for this, including tracing and network traffic. Developers are familiar with this and can jump directly from Visual Studio Code to the relevant point in the test. This is not possible with Cypress.

The selection criteria for a front-end testing tool

When evaluating a testing tool, it is not so much the individual feature that is decisive as the question of which target groups and scenarios it must serve. The decision is based on four criteria.

• Operability and proximity to development: End-to-end testing often originates in the testing department, component testing in development. The tool must be flexible enough for both groups.
• Mocking capabilities: At component level, what counts is how well network traffic and event handling can be simulated.
• Error analysis: How quickly can you find the cause of a test error, i.e. the problematic point in the application?
• Scenario coverage: Multi-tab and cross-domain testing noticeably separate the tools.

WebdriverIO performs significantly weaker in this comparison. The probable reason is of a commercial nature: Playwright is backed by Microsoft, which is interested in integrating it into its own tool landscape, such as Visual Studio Code. Cypress has a commercial product behind it. WebdriverIO is open source without comparably large players, and this visibly slows down the speed of development.

Playwright is the first choice for new projects

Anyone starting a new project today should go straight for Playwright. Cypress would hardly be an option.

Cypress has strengths in the details. The round trip seems subjectively a tad faster. On the other hand, Playwright allows the targeted execution of individual tests, individual test files or several selected tests, while Cypress always processes an entire test file.

Angular support for component testing in Playwright remains an open question. A merge request for this has been stuck with a long thread for some time. However, given the speed of development, this issue may soon be resolved.

If I were to start a project again today, I would go straight to Playwright. I wouldn’t use Cypress at all. Dehla Sokenou

When it’s worth switching and when it’s not

You don’t migrate an existing test suite out of principle, but according to a cost-benefit calculation. If you don’t need the strengths of Playwright in your current project, there is no compelling reason to switch.

A project with hundreds of Cypress tests that does not require multi-tab scenarios and will expire at some point anyway, it makes sense to stay on Cypress. With a long-lasting product, the calculation looks different because the investment is more likely to pay for itself over the longer service life. Start new projects directly with Playwright.

However, there is one technical limitation: Cypress will not overcome its limitation as long as the architecture does not change fundamentally. The one browser window and the problems with cross-domain testing are structural, not a question of individual releases.

Migration according to the pathfinder principle instead of big bang

Running two frameworks in parallel is a viable option, not a problem case. Migration is a step-by-step process instead of a major conversion.

The Pathfinder principle works like this: Whenever you need to touch a test anyway, you decide whether to transfer it now or leave it in the old tool. In this way, the inventory is migrated gradually without anyone having to plan a complete shutdown for a major migration.

Some areas will remain permanently in the old tool, and that’s fine. Component libraries with consolidated modules such as buttons, lists or tables are rarely touched. Where no one passes by, there is no need to migrate. The only important thing is that it remains known that these old tests exist.

The change is easy because both tools are close to development. Tests are often written in TypeScript anyway, analogous to the application. Syntax and semantics differ, but the principle remains similar. If you outsource recurring parts such as the mounting of a component to fixtures, the tests are also aligned.

To try things out, you can start with the end-to-end testing. According to the test pyramid, there are fewer of these at the top than component tests at the base anyway, so the rewriting is manageable. Some of this can even be automated because many constructs are simply transferred from one syntax to another.

When you should watch the market

Monitor the market especially when you start a new project. This is the natural moment to check what is still state of the art and what is outdated.

Not every new bandwagon is worth jumping on. Especially in the frontend, technologies come and go quickly. Tools that are only maintained by a single open source developer quickly end up in a dead end as soon as that developer runs out of time. What feels great in the short term is of no use to you if further development stops.

The better rule of thumb: rely on established technologies, especially for long-lasting projects, but regularly check whether you are still at the cutting edge. If you recognize a dead end, it’s better to react early, even if the conversion costs time and effort. Staying at a dead end is more expensive.

The compatibility of the testing frameworks themselves is remarkably robust. Cypress had a major break a few years ago, which forced a refactoring, but has been stable ever since. Playwright also emphasizes compatibility. The pain today comes more from the frontend: updating to a new Angular or React version is usually more complex than updating the testing framework.