Robot Framework is a generic, technology-agnostic automation framework built on keyword-driven testing. It has no automation technology of its own. Instead, keyword libraries connect it to whatever technology you need, such as Selenium, Playwright or Appium. Keywords can be layered from technical low-level actions up to business-level test steps, so users can write tests without writing Python code.
Key Takeaways
- Robot Framework has more than three million downloads a month and a Slack channel with 32,000 members, which makes it one of the most widely used open source test automation frameworks in the world.
- The Robot Framework Foundation is funded by membership fees from around 77 companies, including Cisco and Nokia, and by the annual RoboCon conference, for a budget of about 250,000 euros a year.
- Robot Framework is technology-agnostic and works with around 700 libraries on PyPI, covering web and mobile tests, database access and even terminal emulation for 3270 systems.
- If you use open source, report bugs, ask questions and improve the documentation: maintainers have no direct customers, so they rely on this feedback to decide what to build next.
- Companies that have built their own test libraries can benefit from open sourcing them, because other companies then pay for features that every user gets at the same time.
What Is Robot Framework?
In testing, Robot Framework sits one level above the tools that actually drive a browser or an app. It doesn’t bring any automation technology of its own. You plug in whatever you need underneath, for example Selenium, Playwright or Appium, which puts it firmly in the test automation toolbox without tying it to one kind of system.
Robot Framework started out as a keyword-driven testing framework, built specifically for acceptance test-driven development. Since then, the pure testing focus has moved into the background. Requests from the community led to the framework now covering RPA and other kinds of automation as well, which is why it is now called a generic automation framework.
How does Robot Framework compare to Cucumber? Structurally, they are quite similar. There is a specification language on top and step implementations underneath in another language, usually Python. The framework runs the parsed text, calls an automated function for each step, checks the return value and logs the result. Through a Gherkin parser, it also supports behavior-driven testing with Given-When-Then and feature files.
How Keyword-Driven Testing Works in Robot Framework
Robot Framework’s strength is that test steps can be stacked in layers. Small steps combine into larger, business-level building blocks.
An example makes this concrete. A low-level keyword like “Set Text” with a locator pointing to the username field can be wrapped into “Set Username”. “Set Username”, “Set Password” and “Click Login” together make up “Login User”. On top of that sits a high-level keyword such as “Log in as administrator”.
This layering has a practical consequence. If someone hands you a ready-made click keyword, you can build something business-level out of it without writing a single line of code. Many people write Robot Framework tests without ever touching Python, because they build on existing libraries.
How Much Technical Know-How Do You Need?
Business experts do well with Robot Framework as long as they have some affinity for technology. The barrier to entry is deliberately low.
The syntax is designed to be readable, much like Gherkin. Keywords and arguments are separated by two or more spaces, with no curly braces or funky special characters. Exactly those syntax hurdles tend to scare business users off quickly.
With business-level and technical high-level keywords available, business users can get a long way. Their involvement drops once custom keywords have to be written in Robot Framework syntax. Anyone who implements their own libraries ends up in Python code, and that’s where you need a typical test automation engineer. Several tools also put a graphical layer on top of the framework and let you assemble test sequences by drag and drop. Behind the scenes, they generate Robot code again.
The Ecosystem Is the Real Superpower
There is a library for almost every use case. In Robot Framework, these adapters are called keyword libraries.
They range from app and web tests to embedded testing and terminal emulation. There is even a 3270 library for testing green-screen mainframe systems. PyPI lists around 700 projects related to Robot Framework, and GitHub recognizes it as a language of its own.
This is where it differs from Selenium or Playwright. Those are, first of all, remote control technologies for the system under test. To turn them into test steps, you need an abstraction layer: the glue code between the driver and the business-level keyword. The Selenium Library, for example, one of the oldest libraries, comes with around 150 technical keywords such as “Type Text”, “Get Title” or “Title Should Be”. A huge library ecosystem has grown out of that history.
Why Robot Framework Is Strong for End-to-End Tests
Robot Framework spans any technology you throw at it, which makes it suitable for true end-to-end tests. That is exactly what web-focused tools can’t do.
Real systems rarely consist of a single application. A desktop app has a server, a database, an API, maybe a mobile device or connected IoT sensors. A tool like Cypress covers only the web frontend and some API testing. It can’t connect to a database, and it can’t test a WPF desktop application.
In Robot Framework, you can combine different adapters and control them through one shared keyword platform: web, desktop, app, database. Before a test, for instance, you can load a database dump and then check the user interface. You stay in one language and still cover every part of the system.
You can plug in your own technologies, too. If you prefer C# to Python, you build a matching adapter and your own libraries for complex test objects with their own APIs. Ready-made libraries for other parts of the system, such as a website, keep working alongside them.
Data-Driven Testing with the DataDriver
Data-driven testing works through the Robot Framework DataDriver. You define a single test and point it to a data source. At runtime, the framework creates a separate test case for each row.
Each of those test cases then shows up individually in the report. Out of the box, the DataDriver reads CSV and XLSX. It can connect to PICT, an open source tool from Microsoft for pairwise testing, and it supports custom data readers. Some users have built readers that pull their test data straight from a database.
The use case comes from practice. Insurance companies often have to run the same test sequence with hundreds of tariff records and many parameters. For databases themselves, there is also an SQL-based Database Library that connects to any database, from DB2 and MySQL to Oracle.
How the Robot Framework Foundation Is Funded
The Robot Framework Foundation carries the further development and is funded by membership fees and the annual conference. Its yearly budget is around 250,000 euros.
The Finnish association was founded in 2015 by six competing consulting companies plus Nokia, who pooled their money. Today the Foundation has around 76 to 77 members, among them Cisco, DB Schenker, NRW Bank, the City of Munich and Fresenius Medical Care.
Fees are tiered. A one-person consultancy pays 500 euros a year. Companies start at 1,500 euros and move up through 3,000 and 6,000 to 12,000 euros. The multiplier is obvious: for every euro a member pays in, the community adds around 70 more. The second source of income is the RoboCon conference in Helsinki.
With that money, the Foundation pays one full-time employee for marketing and organization and two core developers on a freelance basis, one of them Pekka Klärck, who originally created the framework. They review pull requests, check quality standards and maintain the documentation.
A Broad Base Keeps an Open Source Project Alive
A community with many backers is the best risk management an open source project can have. If you bet on a framework, you want to know it won’t disappear tomorrow.
The risk is real. Gauge from ThoughtWorks stopped being funded at some point. SpecFlow is effectively dead and only lives on as a fork. Cucumber Inc. bought the rights to the name and later laid off the people who were building Cucumber. Cucumber survives mainly because its community is too big to fail.
“With around 70 members, you have a massive base. Even if one board member steps down, there are seven others who carry on. And if they leave, new ones come along.”
(René Rohner)
In Robot Framework, the load is spread across many shoulders, on the Foundation board as well as in the volunteer working groups for the syllabus, the website and the documentation. With more than three million downloads a month and a Slack channel with 32,000 members, the project is very much alive.
Contributing to Open Source Starts with Asking
You don’t have to be a great coder to contribute to an open source project. Feedback, bug reports and documentation are often just as valuable as code.
Many users stay silent because of a misplaced sense of restraint. They feel the software was a gift, so they have no right to complain. But maintainers don’t have customers who hand them requirements. They are stuck in their own bubble and often don’t know which feature is missing next. If you point out politely where it hurts, you help the project directly.
Ways to contribute without writing code:
- Ask questions in the Slack channel or in an issue
- Report bugs, an obvious strength if you’re a tester
- Write up feature requests
- Improve the documentation, for example with a pull request, when something was unclear
The Browser Library makes this visible. A contribution bot puts everyone who opens a ticket, reports a bug, contributes code or documentation, or donates money on a wall of fame. That way, hundreds of contributions get recognized.
Give Your Clever Solution Back to the Community
Consulting companies can open source libraries they have built instead of letting them gather dust in the basement. That creates value that multiplies across several clients.
RoboSapiens shows how this works. The library automates SAP GUIs, and one client paid for it in full. By mutual agreement, it was released as open source. A second client found it useful and paid for another feature, which the first client got for free. A third client then followed suit.
If you’re building something you wouldn’t sell anyway, consider open sourcing it. Make sure not to hard-code any company data and keep the solution anonymous. It hurts less than most people fear, and you can get advice on how to do it.
Where Robot Framework Is Heading
Robot Framework has been heavily refactored over the past few years, and its technology base has grown. If you last looked at it in 2018, it’s worth a second look.
Tooling support in particular has improved. For Visual Studio Code, there is a full IDE with step debugging in both Python and Robot code, complete auto-completion and static code analysis with a linter. Better PyCharm integration is in the works, co-funded by the Foundation, because many users from the Python world rely on it.
At the time of the conversation, Robot Framework 8 was planned for the following year, with some API features for advanced users. No big killer feature is planned for the core, and that’s on purpose. The framework is highly decentralized. Real innovation often happens in individual libraries such as the Browser Library, without touching the core.
Frequently Asked Questions
Can business users without programming knowledge automate test cases?
Yes, as long as high-level business and technical keywords are already available. Keywords can be layered: “Set Username,” “Set Password,” and “Click Login” become “Login User,” with the business-level description “Log in as administrator” above it. However, the involvement of business experts decreases as soon as custom libraries become necessary. Then you end up working with Python code and need a test automation specialist.
Is Robot Framework intended only for test automation?
No. Originally, it was a keyword-driven testing framework, explicitly designed for acceptance test-driven development. Requirements from the community have led to it now also covering RPA and other forms of automation, which is why it’s referred to as a generic automation framework. Via a Gherkin parser, it also supports behavior-driven testing with Given-When-Then and feature files.
How does Robot Framework differ from Selenium or Playwright?
Selenium and Playwright are remote control technologies for the system under test; they are not testing frameworks. What’s missing is a layer of abstraction, the “glue code,” between the driver and the business-logic test steps. This is exactly what keyword libraries provide: The Selenium Library, one of the oldest, includes around 150 technical keywords such as “Type Text” or “Title Should Be.” There are nearly 700 projects on PyPI related to the framework.
Is a pure web testing tool sufficient for end-to-end testing?
Usually not, because real-world systems rarely consist of a single application. A desktop app typically includes servers, a database, an API, and sometimes mobile devices or IoT sensors. A tool like Cypress covers only the web front end and some API testing; it doesn’t connect to a database or test desktop WPF applications. In contrast, combinable adapters allow you to test web, desktop, mobile apps, and databases all in one language.
How can you cover hundreds of test records without writing a test case for each one?
Using the Robot Framework Data Driver: You define a test and specify a data source from which a separate test case is generated for each row at runtime. These cases appear individually in the report. By default, the Data Driver reads CSV and XLSX; it can integrate with PICT for pairwise testing, and custom readers can also retrieve test data directly from a database.
How can you tell if an open-source testing framework will be maintained in the long term?
By the breadth of its support. ThoughtWorks’ Gauge eventually ran out of funding; SpecFlow is effectively dead and lives on only as a fork; and Cucumber survives primarily because its community is too large to fail. With Robot Framework, the burden is shared among roughly 77 member companies of the Foundation, three million monthly downloads, and a Slack channel with 32,000 members.
Can you contribute to an open-source project without writing code?
Yes, and often that’s more valuable than code. Maintainers don’t have customers providing them with requirements, so they often don’t know which feature is missing next. Questions in the Slack channel, bug reports, feature requests, and improved documentation help directly. The Browser Library highlights such contributions on a “Wall of Fame” via a contribution bot.
Is it worth publishing an internally developed testing library?
Often yes, because the development costs are spread across multiple customers. The SAP GUI library RoboSapiens was fully funded by one customer and then made open source. A second customer funded an additional feature, from which the first benefited for free, and a third followed suit. It’s important not to include any company data in the code and to keep the solution anonymous.
