Robot Framework is a generic, technology-agnostic automation framework based on the keyword-driven testing approach. It does not have its own automation technology, but connects any technologies such as Selenium, Playwright or Appium via keyword libraries. Keywords can be layered: from technical low-level actions to functional high-level test steps, without users having to write Python code.
Key Takeaways
- Robot Framework has over three million downloads per month and a Slack channel with 32,000 members, making it one of the most widely used open source test automation frameworks in the world.
- The Robot Framework Foundation is funded by membership fees from around 77 companies, including Cisco and Nokia, as well as the annual Robocon conference, resulting in a budget of around €250,000 per year.
- Robot Framework is technology agnostic and can be combined with around 700 libraries on PyPI, from web and mobile testing to database connectivity and terminal emulation such as 3270 systems.
- Anyone using open source should report bugs, ask questions and improve documentation, as maintainers without direct customer contact are dependent on this feedback in order to prioritize meaningful further developments.
- Companies that have developed their own test libraries can benefit from external contributions by open sourcing these solutions: Other companies then fund features that benefit all users at the same time.
What is the Robot Framework?
The Robot Framework is a generic, technology-agnostic automation framework. It does not come with its own automation technology, but docks onto any technology such as Selenium, Playwright or Appium.
The Robot Framework was originally intended as a keyword-driven testing framework, explicitly for acceptance test-driven development. In the meantime, the pure testing focus has faded into the background. Requirements from the community have led to the framework now also covering RPA and other automations. This is why it is now referred to as a generic automation framework.
In terms of structure, the Robot Framework can be compared with Cucumber. There is a specification language and step implementations in another language, usually Python. The framework executes the parsed text, calls an automated function for each step, checks the return value and logs the result. The framework also supports behavior-driven testing with given-when-then and feature files via a gherkin parser.
How does the keyword-driven approach work?
The strength of the Robot Framework lies in the fact that test steps can be built up in layers. Small steps result in larger, technical building blocks.
An example makes this tangible. A low-level keyword such as “Set Text” with a locator on the username field can be combined to form “Set Username”. “Set Username”, “Set Password” and “Click Login” together result in a “Login User”. A high-level keyword such as “Log in as administrator” is then placed above this.
This cascading has a practical consequence: if you get a ready-made click keyword from someone, you can build something technical from it without writing a line of code yourself. Many users write robot framework tests without ever having touched Python because they are based on existing libraries.
How much technical know-how do you need?
Technical experts work well with the Robot Framework if they have a certain affinity for technology. The entry hurdle is deliberately low.
The syntax is designed for readability, similar to Gherkin. Keywords and arguments are separated by two or more spaces, without curly brackets or funky special characters. It is precisely such syntax hurdles that otherwise quickly drive specialist users away.
The availability of professional and technical high-level keywords gets business users a long way. Participation decreases as soon as custom keywords have to be written in robot framework syntax. Those who implement their own libraries end up in Python code and need the typical test automation tool. Several tools are also graphically based on the framework and allow test sequences to be clicked together using drag-and-drop. Robot code is then created behind this.
The ecosystem is the real superpower
There is a suitable library for almost every use case. In the Robot Framework, these adapters are called keyword libraries.
They range from app and web tests to embedded testing and terminal emulations. There is even a 3270 library that can be used to test green screen host systems. There are around 700 projects based around the Robot Framework on PyPI, and it is listed as a separate language on GitHub.
This is where it differs from Selenium or Playwright. These are just remote control technologies for the system under test. In order to build a test step from this, you need an abstraction layer, the glue code between the driver and the technical keyword. The Selenium Library, for example, one of the oldest libraries, has around 150 technical keywords such as “Type Text”, “Get Title” or “Title Should Be”. A huge library ecosystem has grown out of this history.
Why is the Robot Framework strong for end-to-end testing?
The Robot Framework spans an umbrella over any technology and is therefore suitable for real end-to-end testing. This is exactly what web-focused tools cannot do.
Real systems rarely consist of a single application. A desktop app has a server, a database, an API, perhaps a mobile device or connected IoT sensors. A tool like Cypress only covers the web front end and some API testing. It can’t connect a database or test a desktop WPF application.
Different adapters can be combined and controlled via a common keyword platform: Web, desktop, app, database. Before testing, you can import a database dump and then check the interface. You stay in one language and still operate the entire system landscape.
You can also integrate your own technologies. If you prefer C# to Python, you can build a corresponding adapter and your own libraries for complex test objects with their own APIs. Ready-made libraries for other parts of the system, such as a website, can also be used.
How do you do data-driven testing?
Data-driven testing works via the Robot Framework Data Driver. You use it to define a single tester and specify a data source from which the framework generates a separate test case for each line at runtime.
These test cases then end up individually in the report. The Data Driver loads CSV and XLSX as standard. It can connect PICT, an open source tool from Microsoft for pairwise testing, and support its own data readers. Some users have built readers that pull their test data directly from a database.
The use case comes from practice. Insurance companies often have to run the same test sequence with hundreds of tariff data sets and many parameters. For databases themselves, there is also a database library based on SQL, which connects any database from DB2 to MySQL to Oracle.
How is an open source project like this financed?
The Robot Framework Foundation is responsible for further development and is financed by membership fees and the annual conference. The annual budget is around 250,000 euros.
The Finnish association was founded in 2015 by six competing consulting companies plus Nokia, who together put money into a pot. Today, the Foundation has around 76 to 77 members, including Cisco, DB Schenker, NRW Bank, the City of Munich and Fresenius Medical Care.
The contributions are staggered. A one-man consultancy pays 500 euros a year, companies start at 1,500 euros and increase to 3,000, 6,000 and up to 12,000 euros. The leverage is clear: for every euro paid in, around 70 more come from the community. The second source of income is the RoboCon conference in Helsinki.
The Foundation uses the money to pay a full-time employee for marketing and organization as well as two core developers as freelancers, including the original inventor Pekka Klärck. They review pull requests, check quality standards and maintain the documentation.
A broad base protects against the dead horse
A community with many contributors is the best risk management for an open source project. If you rely on a framework, you should know that it won’t stop tomorrow.
The danger is real. Gauge from ThoughtWorks was no longer funded at some point. SpecFlow is effectively dead and only lives on as a fork. Cucumber Inc. bought the naming rights and later laid off the employees who implemented Cucumber. Cucumber survives mainly because its community is too big to fail.
With these 70 or so members, you have a massive base. Even if one board member quits, there are seven others to carry on. And when they leave, new ones join. René Rohner
With the Robot Framework, the load is spread across many shoulders, both on the Foundation Board and in the voluntary working groups for the syllabus, website and documentation. With over three million monthly downloads and a Slack channel with 32,000 members, the project is vital.
Open source contribution starts with asking questions
You don’t have to be a good coder to contribute to an open source project. Feedback, bug reports and documentation are often just as valuable as code.
Many users remain silent out of a false sense of inhibition. They think they have been given the software as a gift and have no right to complain. However, maintainers do not have customers who provide them with requirements. They sit in their own juice and often don’t know which feature will be missing next. If you politely say where the pain points are, you help the project directly.
Concrete contribution paths without code:
- Ask questions in the Slack channel or via an issue
- Reporting bugs, an obvious strength, especially as a tester
- Formulate feature requests
- Improve documentation, for example via pull request, if something was unclear
The Browser Library makes this visible. Via a contribution bot, everyone who opens a ticket, reports a bug, contributes code or documentation or deposits money ends up on a wall of fame. This makes hundreds of contributions visible.
Give the cool solution back to the community
Consulting companies can open-source self-developed libraries instead of letting them gather dust in their own basement. This creates added value that is multiplied across several customers.
RoboSapiens shows the model. The library automates SAP GUIs and was paid for in full by a customer. It was made open-source on the basis of a joint idea. A second customer found it useful and financed a further feature from which the first customer benefited free of charge. A third customer followed suit.
If you’re building something you wouldn’t sell anyway, consider open-sourcing as an option. Be careful not to write any company data directly into the code, but keep the solution anonymous. It hurts less than most people fear and you can get advice on this.
Where is the Robot Framework heading?
The Robot Framework has been heavily refactored and expanded in its technology base over the last few years. If you last looked at it in 2018, you should take a second look.
Tooling support in particular has grown. There is a fully-fledged IDE for Visual Studio Code with step debugging in both Python and Robot code, full auto-completion and static code analysis via Linter. Better PyCharm integration is in the works and is co-financed by the Foundation because many users from the Python world work with it.
Robot Framework 8 is planned for next year, with some API features for demanding users. A big killer feature in the core is not planned, and that is intentional. The framework is highly decentralized. Real innovations are often created in individual libraries such as the Browser Library, without touching the core.
