Introduction of test automation in a company takes place gradually at best. This serves to adapt the techniques and tools to the needs in the company. But also to adapt the processes and, in particular, to align the expectations in the company with the possibilities of test automation. In practice, an introduction in 3 basic phases has proven successful for us, which represent this step-by-step introduction: Pilot, Application and Roll-Out.
Pilot Project Test Automation - Proof of Concept
The pilot project phase includes several activities: the preparation, the implementation and the evaluation of the pilot.
Preparation of the pilot project
During the preparation, the analysis of the ACTUAL status takes place. For this purpose, the process descriptions, tools used and existing dependencies are recorded by analyzing the associated documentation. Furthermore, interviews are typically also conducted with all participants in order to record the lived processes and the comparison to the documentation. The difference is sometimes serious. During this analysis, priority is given to identifying potential for improvement based on the reduction of activities that were previously performed manually. Corresponding activities to raise this potential (e.g. use of tools, need for modeling, test coverage or traceability) are also analyzed. This results in a catalog of criteria that a test automation solution should fulfill as best as possible.
The next step is the market analysis of the available concepts, the associated tools, and the corresponding dependencies based on the previously identified catalog of criteria. In this process, all available solutions are analyzed and evaluated and subsequently compared. Depending on the time and financial possibilities, the pilot phase is then carried out with the best x solutions. Since there are most likely some offers that obviously do not fit the needs of the company, the implementation with all solutions is nonsensical. Similarly, we do not recommend conducting the pilot with only one (the top ranked) tool, as this narrows down the view of the pros and cons. We recommend performing the pilot with the 3 best-placed tools.
Implementation of the test automation pilot project
The implementation of the pilot is now done with the aim of solving a smaller task that takes only a few person-days. The execution itself can be done with the involvement of the subject matter experts. In any case, the experts for test automation should be in charge. The path to the solution, the advantages and disadvantages experienced in the process, and the solution itself are logged for each tool used.
Evaluation of the test automation solution
Following the implementation, the experiences made are evaluated and presented to the entire group of participants in a workshop. Here, the differences between the tools used are re-evaluated. In particular, practical experience in using the tools can revise the previously made assessments. Two fundamental questions are to be clarified during the workshop:
-
- Should the (locally limited) use of test automation in real projects be recommended after completion of this pilot?
-
- Which tool should be used for this operation?
If the first question has been answered positively and an answer has also been found for the second question, then the last milestone of this phase has been mastered and the application phase begins.
Application
The application phase essentially comprises two steps: The selection of the department / group in which the commercial application of test automation will take place, and the application itself.
When selecting the department, one of the essential points is the reuse of the experience from the pilot project and the necessary adjustments to ensure the smoothest possible transition from pilot to application. We recommend every decision maker to pay attention to the selection of the technically suitable persons here as well, since the implementation is still in a sensitive phase. Furthermore, as part of the adaptation, the appropriate tool must be selected and the processes adapted. For the evaluation of success, it is also advisable to carefully select the relevant key figures.
Following the preparation, the application phase takes place. Especially during the first application, it is recommended that the users are in constant exchange with experts in order to be able to discuss and reflect on their experiences, obstacles and progress with them. In the beginning, this is done with training courses via application-accompanying coaching with the aim of independent work in the department.
With the help of the key figures defined at the beginning, the selected test automation technology can be finally evaluated after the application phase with regard to commercial application suitability. This experience should provide enough information to decide whether to roll out the technology company-wide.
Roll-out of test automation
In the roll-out phase, the application of test automation is formalized. The development process is adapted to test automation and a corresponding training program is set up. Responsible persons are appointed in the departments to monitor and control the process improvement. Coaching should be provided to accompany the application, especially when it is used for the first time. The long-term goal of the roll-out is for employees to internalize the process without external support and to adapt it to the changing business environment as required.
Test automation in agile projects
In agile projects, these phases are shortened. Usually, a test automation tool is tested on a subarea of the application during the ongoing project. The result is evaluated in a retrospective, for example, and successively rolled out to the project if successful. Here, the deployment is much more dynamic, tools are changed occasionally, and there is often a tool landscape instead of a single test tool.