Application Value Assessment
Measuring, testing and assessing the quantity, quality and complexity of application systems forms a sound basis for decisions on IT strategy. Particularly in the case of legacy systems with a large, evolved scope of code and a wide variety of languages, data can be determined and comparability created that can be used to evaluate various further development scenarios (maintenance, migration, refurbishment, etc.). To do this, however, it is necessary to select the appropriate key figures and metrics from the very large number available.
Goals of a measurement project
Typical objectives that are pursued with a project to measure application systems are
- Determining the size, complexity and quality of a system’s code
- The development of a metrics database for ongoing quality assurance
- Identifying and estimating alternative further development strategies
In order to be able to fulfill the first objective, the measurement of the system, it is necessary to structure the source code of the application and spread it out for the measurement in such a way that a meaningful comparison of the measurement results is possible. This can be done, for example, by language (Cobol, PL1, C++, etc.) or by specific technical areas. Subsequently, the individual areas can be compared with each other in terms of size, quality and complexity and anomalies can be examined in more detail. The measurement is not limited to program code; databases and user interfaces can also be measured.
The creation of a metrics database enables the ongoing measurement of the system over time. This makes it possible to observe whether, for example, the overall quality has improved or deteriorated in the event of changes or extensions to the system. Ideally, ongoing measurement is embedded in the quality assurance process.
The results of the measurement provide a uniform basis for evaluating various further development scenarios and estimating their cost. This can be used as a basis for the future IT strategy. Possible scenarios are
- Annual maintenance of the system
- New development of the system
- migration
- Refurbishment
- Encapsulation
Selection of metrics
A prerequisite for every measurement project is the selection of suitable metrics. This selection is also the first step in the ISO9126 measurement process. However, the literature provides little guidance on which metrics to select. Software systems are complex constructs with many different properties that can be measured. In his book “Software Complexity Metrics”, Horst Zuse alone identified more than 300 metrics. Each metric measures a different property, e.g. the McCabe metric measures internal process complexity, the Halstead metric measures language complexity or the function point metric measures the interactions between a system and its environment. Ultimately, it is a question of the objective of the measurement project and the type of system as to which metrics are most suitable.
In the measurement projects carried out by one of the authors over the last 20 years, experience with a wide variety of systems has resulted in a compilation of size, complexity and quality metrics that have proven their worth many times over in the analysis and evaluation of application systems. These include complexity metrics such as data complexity, control flow complexity and language complexity, but also quality metrics such as those for measuring portability, maintainability, conformity and testability.
The measuring process
The measurement process for carrying out the measurement, testing and evaluation of the system typically consists of the following steps:
- selection of metrics
- structuring of the system to be measured
- configuration of the tools, adaptation to local conditions (e.g. own language constructs)
- execution of the measurement
- transfer of the measured values to a metrics database
- evaluation of the measurement results
- implementation of the estimates
Results
In addition to the metric reports on the size, quality and complexity of the system and the subsystems/components as well as the effort estimates for various further development strategies, evaluations can be created to prepare the information for different target groups. This is done, for example, in the form of scorecards that relate the qualities and complexities to each other, or in management dashboards to obtain an up-to-date overview of the system.
Frequently Asked Questions
Why is measurement particularly valuable for legacy systems that have evolved over time?
Because with a large codebase and a wide variety of programming languages, any assessment would otherwise remain merely an opinion. Measurement, testing, and evaluating quantity, quality, and complexity provide data and enable comparability between different parts of the system. On this basis, further development scenarios—such as maintenance, migration, or modernization—can be evaluated against one another, rather than selected based on gut feeling.
How can measurement results from different parts of an application system even be compared?
Before measurement, the source code must be structured and prepared in such a way that meaningful comparability is achieved. It is categorized, for example, by languages such as COBOL, PL/1, or C++, or by functional areas. The areas are then compared based on size, quality, and complexity, and any anomalies are examined more closely. Not only program code but also databases and user interfaces can be measured.
What are the advantages of a metrics database over a one-time measurement?
It allows for ongoing measurement of the system over time. When changes or enhancements are made, it becomes clear whether overall quality has improved or deteriorated. The benefit is particularly evident when recurring measurements are firmly embedded in the quality assurance process rather than remaining a one-time snapshot.
Which development scenarios can be weighed against one another using measurement results?
Typically, the following are considered: annual system maintenance, a new development, a migration, a refactoring, and encapsulation. The measurement results provide a uniform basis for evaluation and estimation for all these options. This makes it possible to compare the effort involved, and the decision can serve as the foundation for future IT strategy.
Is there a standard set of metrics for evaluating software?
No. Which metrics are appropriate depends on the objectives of the measurement project and the type of system. The literature offers little guidance on this: In his book “Software Complexity Metrics,” Horst Zuse identified more than 300 metrics alone. In 2009, this selection also formed the first step in the ISO 9126 measurement process.
What do McCabe, Halstead, and Function Points measure, respectively?
Each of these metrics focuses on a different characteristic. The McCabe metric captures internal flow complexity, the Halstead metric captures language complexity, and the Function Point metric captures the interactions between a system and its environment. Anyone who measures only one of these metrics describes, accordingly, only a subset of the system.
What types of metrics should be included in an evaluation of application systems?
A combination of size, complexity, and quality metrics has proven effective. On the complexity side, these include data complexity, control flow complexity, and language complexity. On the quality side, metrics such as portability, maintainability, compliance, and testability are measured. This selection is based on measurement projects conducted by one of the authors over the approximately twenty years leading up to 2009.
How do you prepare measurement results for different target audiences?
In addition to metric reports on the size, quality, and complexity of the overall system and its components, as well as effort estimates, analyses tailored to specific target audiences are produced. Scorecards put quality and complexity into relative perspective. Management dashboards provide an up-to-date overview of the system’s status without requiring recipients to know the individual metrics in detail.
