The integration testing 

 November 1, 2012

Like it?
Share!

From design and architecture to component and system integration

The integration testing has always been neglected in the literature. On the one hand, it is of great importance in the practice of software testers, but on the other hand this link between component testing and system testing is unknown territory for many developers and testers.

In this book, you will get to know the integration testing and its various intermediate stages from member integration, class and module integration to component and subsystem integration to system integration. For each of these integration stages, the book presents field-tested test procedures and test tools and explains them using real practical examples. One section of the book is devoted to each of the different system architectures that are widely used in practice, such as strict and non-strict layer architectures, client/server, and SOA. Their advantages and disadvantages with regard to integration testing are explained and the underlying platforms or technologies are characterized.

Authors: Mario Winter, Mohsen Ekssir-Monfared, Harry M. Sneed, Richard Seidl, Lars Borner

Publisher: Carl Hanser Verlag

ISBN: 978-3-446-42564-4

Edition: 1st edition

Available in bookshops and on Amazon

Table of Contents

1 Introduction

1.1 What is it about?
1.2 integration testing in practice today
1.3 A little history
1.4 integration testing and software quality
1.5 Who is this book written for?

2 Introductory case study

2.1 The old order processing system
2.2 The new order processing system
2.3 integration testing of order processing
2.3.1 Class integration test
2.3.2 Component integration test
2.3.3 System integration test
2.4 Summary

3 Basic information about software testing

3.1 What is testing?
3.2 Black-box methods
3.2.1 Equivalence class test
3.2.2 Boundary value analysis
3.3 White-box methods
3.3.1 Control flow-based test
3.3.2 Data flow-based test
3.3.3 Condition test
3.4 How good is the test?
3.4.1 Measuring Test Coverage
3.4.2 Mutation Testing
3.5 Testing Stages
3.5.1 Module or Component Testing
3.5.2 integration testing
3.5.3 system testing
3.5.4 acceptance testing
3.6 Test Execution and Test Environment
3.7 Testing Process

4 Basics of the integration test

4.1 What is Integration?
4.2 Building Blocks of Integration
4.3 Dependencies
4.3.1 Elementary Building Blocks
4.3.2 Compound Building Blocks
4.3.3 Syntactic and Semantic Dependencies
4.4 Goals, Focus and Pragmatic Definition of Integration Testing
4.5 Stages of Integration
4.6 Procedure in the integration testing

5 Modeling in the integration testing

5.1 Structure Modeling
5.1.1 Objects
5.1.2 Classes
5.1.3 Packages and Components
5.1.4 Physical Structure and Deployment
5.2 Behavior Modeling
5.2.1 Use Cases
5.2.2 Activities
5.2.3 Interactions
5.2.4 States and Reactive Behavior
5.3 Graph Theory
5.3.1 Basic Concepts
5.3.2 Dependency Graphs
5.3.3 Paths and Graph Matrices
5.3.4 Directed Acyclic Graphs
5.3.5 Control Flow and Data Flow Graphs

6 Software dependencies

6.1 Syntactic Dependencies
6.1.1 Visibilities and Interfaces
6.1.2 Building Blocks as Types
6.2 Interaction Dependencies
6.2.1 Accesses to Variables
6.2.2 Operation Calls
6.2.3 Use of Operation Parameters
6.2.4 Contracts Between Building Blocks
6.2.5 Remote Procedure Calls
6.2.6 Message-Based Communication
6.2.7 Event-Based Communication
6.3 Inheritance Dependencies
6.3.1 Generalization, Substitutability, and Polymorphism
6.3.2 Generalization and Interfaces
6.3.3 Horizontal and Vertical Dependencies
6.4 Indirect Dependencies
6.4.1 Data-Oriented Dependencies
6.4.2 Service-Oriented Dependencies
6.4.3 Natural Language Communication and REST Dependencies

7 Types of integration errors

7.1 Errors and Dependencies
7.1.1 Interaction Errors
7.1.2 Inheritance Errors
7.1.3 Indirect Dependency Errors
7.2 Overview of Integration Errors
7.3 Errors in Integration within an Application
7.3.1 Member Integration Errors
7.3.2 Module Integration Errors
7.3.3 Class Integration Errors
7.3.4 Component and Subsystem Integration Errors
7.4 Errors in Integration of Multiple Applications
7.4.1 Application Integration
7.4.2 Enterprise Integration Errors
7.5 Causes of Errors
7.5.1 Different Interface Descriptions
7.5.2 Different Interface Versions
7.5.3 Different Value Ranges
7.5.4 Different Data Interpretation
7.5.5 Lack of Synchronization
7.5.6 Timing Dependencies
7.6 Database Errors
7.7 Integration Error Distribution

8 case studies on the integration testing

8.1 Securities Settlement System
8.1.1 Division of the System into Subsystems
8.1.2 Integration of Subsystems
8.1.3 Testing Batch Processes
8.1.4 End-to-End Testing
8.1.5 Overall System Testing
8.1.6 Summary
8.2 Fee collection system
8.2.1 Risk-based testing approach
8.2.2 Result testing problem
8.2.3 Selective regression testing
8.2.4 integration testing versus subsystem testing
8.2.5 Inference
8.3 Distributed Web Applications
8.3.1 Testing Subsystem Interactions
8.3.2 Two-Stage Testing - Interfaces and Interactions
8.3.3 Lessons Learned integration testing
8.4 Encapsulated Web Services
8.4.1 Testing Client Processes
8.4.2 Testing Web Services
8.4.3 Testing Client Processes with Web Services
8.4.4 Differentiation between Integration and system testing
8.5 integration testing of Software Measurement Tools
8.6 Summary of Case Studies

9 Integration levels

9.1 Member-integration testing
9.1.1 Objectives
9.1.2 Test Basis and Test Objects
9.1.3 Execution
9.1.4 Coverage Measures and End Criteria
9.1.5 Result
9.1.6 Hints and Recommendations
9.2 Class or Module Integration Test Module Integration Test
9.2.1 Objectives
9.2.2 Test Basis and Test Objects
9.2.3 Execution
9.2.4 Coverage Measures and End Criteria
9.2.5 Result
9.2.6 Notes and Recommendations
9.3 Component or Subsystem Test Subsystem integration testing
9.3.1 Objective
9.3.2 Test basis and test objects
9.3.3 Execution
9.3.4 Coverage measures and end criteria
9.3.5 Result
9.3.6 Notes and recommendations
9.4 System Integration Testing
9.4.1 Objective
9.4.2 Test Basis and Test Objects
9.4.3 Implementation
9.4.4 Coverage Measures and End Criteria
9.4.5 Result
9.4.6 Notes and Recommendations
9.5 Extended V-Modell for Application Landscapes: The V-Modell++
9.5.1 Notes on System Integration Testing
9.5.2 Multisystem Testing
9.5.3 Multisystem Acceptance Testing

10 Integration strategies

10.1 Integrating the Building Blocks of an Application
10.1.1 Basics
10.1.2 Stepwise Integration with Stubs and Drivers
10.1.3 Dependency Cycles
10.2 Structure-Dependent Strategies
10.2.1 Strategies for Hierarchical Building Block Structures
10.2.2 Top-Down Strategy
10.2.3 Bottom-Up Strategy
10.2.4 Sandwich strategy
10.3 Structure-independent strategies
10.3.1 Ad-hoc strategy
10.3.2 Critical building blocks strategy
10.3.3 Critical dependencies strategy
10.3.4 Use case-based strategy
10.3.5 Thread-based integration
10.4 Individual strategies
10.4.1 Clustering strategy
10.4.2 Heuristic approaches
10.4.3 Graph-based approaches with cycle resolution
10.5 Object-oriented strategies
10.5.1 Inheritance-based approaches
10.5.2 Inheritance- and interaction-based approaches
10.6 Factors influencing integration strategy
10.6.1 Project environment factors
10.6.2 Test Environment Factors
10.6.3 Architecture and Design Factors
10.6.4 General Factors
10.6.5 Strategy Selection
10.7 Multiple Application Integration
10.7.1 Interaction-Based System Integration
10.7.2 Front-End Integration
10.7.3 Back-End Integration
10.7.4 Service-Based Integration

11 Integration test process

11.1 The Fundamental Test Process
11.1.1 Roles
11.1.2 Test Planning and Control
11.1.3 Test Analysis and Test Design
11.1.4 Test Realization and Test Execution
11.1.5 Evaluation of Initial Criteria and Report
11.1.6 Completion of Test Activities
11.2 Roles in the Integration Test Process
11.2.1 Integration Test Manager
11.2.2 Integration Test Designer
11.2.3 Integration Tester
11.3 Activities in the Integration Test Process
11.3.1 Test Planning and Control
11.3.2 Test Analysis and Test Design
11.3.3 Test Realization and Test Execution
11.3.4 Evaluation, Report, and Closure
11.4 Integration Test Level-Specific Features
11.4.1 Member-integration testing
11.4.2 Class or Module Integration Test
11.4.3 Component Integration Test
11.5 Integration into the Development Process
11.5.1 Waterfall Model
11.5.2 V-Model
11.5.3 Incremental/Iterative Development
11.5.4 Agile approaches
11.6 Special features in the system integration testing
11.6.1 Planning and control
11.6.2 Analysis and design
11.6.3 Implementation and execution
11.6.4 Execution and logging
11.6.5 Evaluation and reporting
11.6.6 Closure
11.6.7 System dependencies and introduction of release management

12 Static analyses

12.1 Interface Types
12.1.1 Unstructured Data Transfer
12.1.2 Common Global Data Areas
12.1.3 Operation Calls and Parameters
12.1.4 Files
12.1.5 Databases
12.1.6 Messages in Distributed Systems
12.1.7 Interface Definition Languages
12.1.8 Extended Markup Language (XML)
12.1.9 Web Service Definition Language (WSDL)
12.1.10 Structured Query Language (SQL)
12.2 Approaches to Static Analysis of Interfaces
12.2.1 Reviews of Interfaces
12.2.2 Inspections of Interfaces
12.2.3 Automated Interface Testing
12.2.4 Automated reconciliation of interfaces
12.3 Prerequisites for static interface analysis
12.3.1 Testability of interface definitions
12.3.2 Readability of interface definitions
12.3.3 Verifiability of interfaces
12.4 Tools for static analysis of interfaces
12.4.1 Interface Verifier
12.4.2 Interface Verifier
12.4.3 Interface Generator
12.4.4 Interface Validator
12.5 Experience with Static Analysis of Interfaces
12.5.1 IDL Analysis
12.5.2 XML Analysis
12.5.3 WSDL Analysis

13 Functional and value-based test design procedures

13.1 Use Case Based Testing
13.1.1 Test Objectives
13.1.2 Prerequisites
13.1.3 Procedure
13.1.4 Discoverable Errors
13.1.5 Integration Test Levels
13.1.6 Known Limitations and Problems
13.2 End-to-End Testing
13.2.1 Test Objectives
13.2.2 Prerequisites
13.2.3 Procedure
13.2.4 Discoverable bugs
13.2.5 Integration test levels
13.2.6 Known limitations and issues
13.3 Parameter-based testing
13.3.1 Test objectives
13.3.2 Prerequisites
13.3.3 Procedure
13.3.4 Discoverable bugs
13.3.5 Integration test levels
13.3.6 Known limitations and issues
13.4 State-Based Testing
13.4.1 Test Objectives
13.4.2 Prerequisites
13.4.3 Procedure
13.4.4 Discoverable Errors and Coverage Criteria
13.4.5 Applicable Test Levels
13.4.6 Limitations and Issues
13.5 Association-Based Testing
13.5.1 Test Objectives
13.5.2 Prerequisites
13.5.3 Procedure
13.5.4 Detectable errors and coverage criteria
13.5.5 Applicable test levels
13.5.6 Limitations and problems
13.6 Generalization-based and pairwise testing
13.6.1 Test goals
13.6.2 Prerequisites
13.6.3 Procedure
13.6.4 Detectable errors and coverage criteria
13.6.5 Applicable test levels
13.6.6 Limitations and problems

14 Sequence-related test design procedures

14.1 Overview
14.2 Control Flow Based Testing
14.2.1 Test Objectives
14.2.2 Prerequisites
14.2.3 Procedure
14.2.4 Discoverable Errors and Coverage Criteria
14.2.5 Applicable Test Levels
14.2.6 Limitations and Problems
14.3 Data Flow Based Testing
14.3.1 Test Objectives
14.3.2 Prerequisites
14.3.3 Procedure
14.3.4 Discoverable errors and coverage criteria
14.3.5 Applicable test levels
14.3.6 Limitations and problems
14.4 Interaction-based testing
14.4.1 Test objectives
14.4.2 Prerequisites
14.4.3 Procedure
14.4.4 Discoverable errors and coverage criteria
14.4.5 Applicable test levels

15 Failure-based, experience-based, and other test design techniques

15.1 Fault-based Testing
15.1.1 Test Objectives
15.1.2 Prerequisites
15.1.3 Procedure
15.1.4 Detectable Faults
15.1.5 Integration Test Levels
15.1.6 Limitations and Problems
15.2 Exploratory Testing
15.2.1 Test Objectives
15.2.2 Prerequisites
15.2.3 Procedure
15.2.4 Detectable Faults
15.2.5 Integration Test Levels
15.2.6 Limitations and Problems
15.3 Further Test Design Procedures

16 Non-functional integration tests

16.1 Test against the quality characteristics
16.2 Internal and external quality characteristics
16.3 Safety test
16.3.1 Functional safety test
16.3.2 Technical safety test
16.3.3 Procedure
16.4 Time and consumption behaviour
16.4.1 Load and Performance Testing
16.4.2 Procedure
16.4.3 Load Generation and Tool Selection
16.4.4 Load and Performance Testing Objectives
16.5 integration testing Distributed Systems
16.5.1 Validation Points in Distributed Systems
16.5.2 Concurrency and Synchronization Testing

17 Integration Test Environment

17.1 Definition of the Test Environment
17.2 Purpose of the Test Environment
17.3 Importance of the Test Environment for the integration testing
17.4 Management Processes for the Test Environment
17.5 Planning and Management of the Test Environment
17.6 Effort Estimation for the Test Environment
17.7 Big Picture = System City Plan
17.8 Test Infrastructure Plan
17.9 Procurement of Hardware
17.10 System installation and configuration plan
17.11 Simulators and emulators
17.12 Stubs and mock objects
17.13 Test drivers and monitors
17.13.1 Observation and control points
17.14 Test tools
17.15 Test data
17.16 Responsibilities
17.17 Release management
17.18 Test environment for maintenance and support
17.19 Dismantling the test environment
17.20 Documentation

18 Integration test automation and dynamic analyses

18.1 Overview
18.2 Automation of Interface Analysis
18.2.1 Purpose of an Interface Analyzer
18.2.2 Functionality of an Interface Analyzer
18.2.3 Prerequisites for an Interface Analyzer
18.2.4 Results of an Interface Analyzer
18.3 Automated Determination of Integration Test Cases
18.3.1 Purpose of a Test Case Generator
18.3.2 Functionality of a Test Case Generator
18.3.3 Prerequisites for Test Case Generation
18.3.4 Results of a Test Case Generator
18.4 Automated Generation of Interface Data
18.4.1 Purpose of an Interface Generator
18.4.2 Functionality of an Interface Generator
18.4.3 Prerequisites for Interface Generation
18.4.4 Results of an Interface Generator
18.5 Test Drivers for Integration Test Control
18.5.1 Purpose of a Test Driver
18.5.2 Functionality of a Test Driver
18.5.3 Prerequisites for Integration Test Execution
18.5.4 Results of Integration Test Execution
18.6 Dynamic Analysis of Interface Usage
18.6.1 Purpose of an Interface Validator
18.6.2 Functionality of an Interface Validator
18.6.3 Prerequisites for Interface Validation
18.6.4 Results of Interface Validation
18.7 Dynamic Analysis of Integration Test Procedures
18.7.1 Purpose of a Traceability Tool
18.7.2 Functionality of a Traceability Tool
18.7.3 Prerequisites for Test Traceability
18.7.4 Results of Test Traceability
18.8 Integration Test Management Tools
18.8.1 Purpose of a Test Control Desk
18.8.2 Functionality of Test Control Desk
18.8.3 Prerequisites for Test Control Desk
18.8.4 Results of a Test Control Desk

19 Outlook

Do you like this post? Share it: