Quality assurance must not only be effective, but also efficient. Automation is the means of choice for increasing efficiency here and has long since outgrown its infancy. However, new challenges for quality assurance and test automation in particular are not waiting in the wings: higher efficiency, better integration into agile and continuous approaches, more complex systems with more interfaces paired with requirements for safety, security and robustness are just a few aspects that test automators must face today.
The basics of software development include not only the design and writing of the software, but also the quality assurance of it. Testing is the most commonly used means of quality assurance and compares the actual behavior of the software with the expected behavior.
However, this testing can be very costly. In particular, the effort required for regression tests after minor changes in the software is comparatively high: Regression tests must be executed, for example, to prove that changes in the software do not have any undesirable side effects. Since this involves testing functionality that is not supposed to be affected by the changes, it is easy to see that the proportion of tests to be executed in the total quantity of tests exceeds the proportion of software changes in the total quantity of software. Accordingly, the test can become costly compared to the software creation, especially if it is performed manually.
Since the execution of tests and especially regression tests is often repetitive, the automatic execution of these tests is an obvious way to increase efficiency.
From freestyle to duty
For a long time, automated test execution was an add-on to manual test execution - sometimes even just a gimmick: Suitable test cases were selected and, with the help of automation tools, corresponding test scripts were recorded or implemented. This worked more or less well and was rather less than maintainable in the long run. Often on the initiative of individuals, solutions were created so that tools and test scripts could continue to be used.
These times are behind us. Not least due to paradigm shifts such as agile software development or the increasing complexity of the developed systems, test automation is indispensable today. In many projects, tests are automated across all levels with a wide variety of tools and the test coverage is successively increased. This has also led to a significant leap in tool support: Test automation tools today are more stable, offer more convenience, provide easier maintainability of test scripts, and are generally more adaptable to the needs of test automators.
Test automation is more than test execution
The term test automation is often equated with automated test execution. Of course, there is also automated test design. In order to keep the maintenance costs for the test scripts manageable, which also increase with the number of test cases, the test design must also be automated: Additional abstraction levels abstract from non-essentials - test scripts can be generated automatically on the basis of models.
However, the potential that automation offers for testing today is far from exhausted - countless activities can be found throughout the entire process that can be created, executed or tested automatically or partially automatically. The boundaries between testing, quality assurance and development are becoming increasingly blurred. A few examples can be found in the box.
Test automation as an integration factor
In our experience, agile projects in particular present an interesting picture. Here, test automation must be practiced from the beginning, because otherwise a high quality of the software cannot be maintained. However, test automation still has a social aspect here, because the necessary formalization in advance brings different, even (previously) autonomously working participants to exchange ideas about a common topic earlier than before - especially when hybrid project structures prevail, where, for example, the business department is not part of the team. In this way, test automation can even be seen as a team-building measure.
Conclusion
As of 2018, test automation has clearly outgrown its infancy, even if there is still plenty of room for improvement. Test automation now manages to deliver on the promises of efficiency and repeatability, and has become much easier and more maintainable to use. Last but not least, test automation can be used in projects to bring different roles into contact and into conversation, which until now have acted rather autonomously. For example, to determine where automation makes sense from a risk perspective or contributes to value.
However, even though automation support has improved significantly in recent years, there are still pitfalls lurking: incompatibilities, interface problems, unstable environments, or lack of data quality can drive up the cost of creating and maintaining test scripts. Thus, the benefits and risk assessment must always be kept in mind so that test automation does not become an end in itself.
The next few years will be very exciting here. The use of machine learning and artificial intelligence opens up completely new fields and applications that are only just beginning to be exploited. The fact is that these mechanisms will be needed to keep the quality of the software and the data high. Because one thing has already happened today: Often no one overlooks the complete software, the architecture or the required data anymore. Intelligent systems are needed here that present this information to us in a comprehensible way and automatically uncover contradictions.
Examples of activities that can be automated
Static Analysis:
- Code Analysis
- Artifact Checks
- Refactoring support
Test Data Management:
- Generating test data
- Anonymization of production-related test data
- Filling and emptying databases
- Archiving test data
Test environment:
- Operating system setup
- Virtual Machine Configuration
- (Re)establishing a defined state of the test environment
Provisioning:
- Quality Gates
- Build Pipeline
- Deployments and automated tests
Documents:
- Generating reports
- Evaluate metrics
- Requirements analysis
Non-functional testing:
- Security tests
- Load and performance tests
- Stress tests
- Usability testing
The article was published in the 01/2018 issue of German Testing Magazin.