Five test dimensions help you structure integration testing and spot the gaps: test objectives, test objects, test level, test basis and typical defects. Map your own tests along these dimensions and you see where integration is already well covered, and where defects can still slip through unnoticed.
The Duden, Germany’s standard dictionary, defines integration as the “(re)creation of a unit (from differentiated parts); completion”. In software development, the forms it takes could hardly be more varied: horizontal, vertical, microservices, APIs, loosely coupled, layers, silos, tightly interlinked, via an integration platform or file-based, encapsulation and so on. Integrating all of this serves just as many goals. On top of that come integration-specific aspects such as the integration strategy, including requirements for the test environment. Keeping track of it all takes structure.
Dimension: Test Objectives
A typical test objective of integration testing is to show that the objects being integrated communicate with each other correctly. As so often, the main point is to reduce the risk of undetected defects and to find failures early.
Functional aspects usually take center stage. But non-functional aspects count as well: reliability, usability, information security, compatibility, portability, modifiability, performance efficiency (see ISO 25010). Depending on the industry, the purpose of the application and customer requirements, they move further into focus. When they do, they need to be taken into account accordingly.
Dimension: Test Objects
The test object determines what the tests and the test environment look like: interfaces, services, APIs, databases, subsystems, but also infrastructure and hardware.
The individual integration objects should already have been tested as a product or subsystem beforehand, independently of the communication at the interfaces. Otherwise, when something fails, you cannot tell whether the problem comes from the interface or from a defect in the component itself. Testing them first saves a lot of time when tracking down defects.
Dimension: Test Level
Depending on the test object, integration testing takes place at a different level, at a different level of abstraction:
- Units: Here, the development environment or framework usually provides good test support.
- System components: Embedded libraries or databases support testing.
- Systems: Even when the interfaces of software systems are well documented, integration is complex and error-prone.
- Integration of software and hardware: The particular challenge for testing here is to take non-functional aspects into account as well.
- Integration of software and data: Both describe information that has to fit together. Usually the software describes a generic part and the data the project-specific part. What matters here is striking the balance between generic and project-specific checks.
Things can get complex across levels, too. Sometimes integration happens at system level (often black box) but targets properties that only show up from a white-box perspective. Striking that balance is a real challenge for many development teams.
Either way, integration requires different teams from different levels to work together, each with its own perspective. That is the only way integration can be efficient.
Dimension: Test Basis
The test basis can include, for example: interface specifications, definitions of communication protocols, sequence diagrams, models such as state diagrams, architecture descriptions, software and system designs, workflows, use cases or descriptions of data structures.
As a rule, the higher the degree of formalization, the more you can rely on the results. If the interface specifications of the communicating products are demonstrably consistent with each other, a lot has already been gained.
Dimension: Typical Defects
Integration tests can uncover many kinds of defects. Typical ones are incorrect data structures, faulty interfaces, wrong assumptions about the data being passed, and missing data. Then there are problems with performance or security (encryption).
What Combining the Test Dimensions Means
These dimensions can be combined in almost any way, which opens up a wide field of possible integration tests. Component and system tests tend to be fairly homogeneous. Integration tests, by contrast, vary widely in implementation, technology and methodology. Many tests, non-functional ones in particular, can only be done meaningfully with automation. The sheer number of combinations is another argument for raising efficiency through test automation.
Setting up test automation is a major challenge in its own right, though. It often calls for adjustments to the frameworks or for in-house test tools. That effort should not be underestimated.
Start Small: Map Out Your Own Test Field
Nobody has to cover every combination at once. To start, a grid of the most important dimensions is enough, for example test level and typical defects. Fill in which integration tests you already have. The empty cells show where defects such as incorrect data structures or missing data can still slip through today. That is where your next improvement begins.
Frequently Asked Questions
What is the purpose of integration testing?
Integration testing verifies that the objects to be integrated communicate correctly with one another. There are two main objectives behind this: to minimize the risk of undetected errors and to identify the failures early on. Functional aspects are usually the primary focus, but depending on the industry, intended use, and customer requirements, non-functional characteristics may take on greater importance and should then be deliberately taken into account.
Which non-functional characteristics should be tested during integration?
Reliability, usability, information security, compatibility, portability, modifiability, and performance efficiency. The 2019 article cited ISO 25010 as a reference standard. The weight given to these characteristics depends on the industry, intended use, and customer requirements. Typical issues in this area include performance problems and weaknesses in encryption.
Why should the individual components already be tested before integration testing?
Because otherwise, in the event of an error, it is no longer possible to determine whether a problem stems from communication at the interface or from an error within the component itself that is unrelated to the interface. If the integration objects have been tested beforehand as a product or subsystem, this saves a great deal of time later during troubleshooting. This precondition is crucial to the success of integration testing.
How do integration tests at the unit level differ from the integration of entire systems?
At the unit level, there is usually good support from the development environment or framework; for system components, integrated libraries and databases provide assistance. When integrating entire software systems, the task remains complex and error-prone, even if the interfaces are well documented. It becomes particularly challenging when integrating software with hardware or with data.
What elements serve as the test basis for integration testing?
Interface specifications, definitions of communication protocols, sequence and state diagrams, architectural descriptions, software and system designs, workflows, use cases, and descriptions of data structures. The higher the degree of formalization, the more reliable the test results are likely to be. Even the assurance that the interface specifications of the communicating products are consistent with one another is a major benefit.
What types of errors are typically only discovered during integration testing?
Incorrect data structures, faulty interfaces, incorrect assumptions about the data being passed, and missing data. In addition, there are issues related to performance efficiency and security, such as those related to encryption. These types of defects arise from the interaction between components and therefore often go undetected when individual objects are tested in isolation.
Can integration testing be conducted effectively without test automation?
Only to a limited extent. Many tests, especially non-functional testing, can only be meaningfully tested through automation, and the large number of possible combinations of test objectives, test objects, and levels makes it necessary to increase efficiency through automation. Setting up the automation itself, however, is time-consuming: adjustments to frameworks or custom test tools are often required.
