UML Design Quantity Metrics
Design quantity metrics are counts of the diagram and model types contained within the UML model. The model types are further subdivided into design entity counts and design relationship counts 1.
The 9 design diagram type counts are:
- Use Case Diagrams
- Activity Diagrams
- Class Diagrams
- Sequence Diagrams
- Interaction Diagrams
- State Diagrams
- Component Diagrams
- Distribution Diagrams
- Design Diagrams in all
The 17 design entity type counts are:
- Use Cases
- base/super classes
- Design Entities in all
- Design Entities referenced
The 10 design relationship type counts are:
- Class Hierarchy Levels
- Method Invocations
- Activity Flows
- State Transitions
- Test Cases
- Design Relations in all
These quantities or element counts have been selected on the basis of their relation to the goals of object-oriented system design in accordance with the goal-question-metric method of Basili and Rombach 2.
UML Design Complexity Metrics
Design complexity metrics are computations for calculating selected complexities 3. Complexity is defined here as the relation of entities to relationships. The size of a set is determined by the number elements in that set. The complexity of a set is a question of the number of relationships between the elements of that set. The more connections or dependencies there are relative to the number of elements, the greater the complexity 4. The complexity of a single entity is determined by the number of sub entities relative to the number of relationships between those sub entities. Overall design complexity can be simply stated as:
Bearing this in mind, the following complexity types have been defined for UML models.
Object Interaction Complexity
The more interactions between objects and the more associations between classes there are, the higher will be the complexity. In this way both the abstract level of the class as well as the physical level of the objects is taken into consideration. This measure is an inverse coupling metric. It is based on empirical evidence that systems with many dependencies among their parts are difficult to maintain 5.
Class Hierarchical Complexity
The more hierarchical levels there are in the class hierarchies, the more dependent the lower level classes are on the higher level ones. Deep inheritance has often been criticized for leading to increased complexity. This metric corresponds to the depth of tree metric from Chidamer and Kemerer 6. It is based on empirical evidence that object-oriented systems with deep inheritance trees (e.g.> 3) are more error prone than others.
Class Data Complexity
The more data attributes a class has the higher its complexity. This corresponds to the class attribute metric in the Mood metrics 7. The design goal is to have many classes, each with a few data attributes, as opposed to having a few classes, each with many attributes. This goal is based on the assumption that it is easier to test and maintain smaller sets of data.
Class Functional Complexity
The more methods, i.e. functions, a class has, the higher its complexity, whereby it is assumed that each class has at least two implicit functions – a constructor and a destructor. This corresponds to the Number of Methods metric of Chidamer and Kemerer 6. The design goal is to have many classes, each with a minimum number of functions, as opposed to having a few classes, each with many methods. This goal is based on the assumption that it is easier to maintain and test a system which is broken down into many small chunks of functionality.
Object State Complexity
Objects are instances of a class. Objects have states. The more they have, the more complex they are. A simple class is a singleton with one object that has a static state. A complex class is one with multiple objects, each with several potential states. Neither the CK nor the MOOD metrics consider state complexity, even though it is a principle driver of test effort together with the cyclomatic complexity of the methods. The design goal is to have many classes, each with a minimum number of functions, as opposed to have as few object states as possible, but this is determined by the application. If an object such as an account has many states, e.g. opened, balanced, overdrawn, suspended, closed, etc., they all have to be created and tested.
State Transition Complexity
The connection lines of a state diagram represent the transitions from one state to another. A given state can have any number of successor states. The more there are, the higher the complexity of the state transition graph. As with the McCabe cyclomatic complexity measure, we are actually measuring here the relation of edges to nodes in a graph 8. Only here the nodes are not statements but states and the edges are not branches but transitions. The design goal is to have as few transitions as possible, since every state transition has to be tested at least once and that drives the test costs up.
Activity Control Flow Complexity
The connection lines of an activity diagram represent the flow of control from one activity to another. They can be conditional or non-conditional. Conditional flows add to the complexity of the process being modeled. An activity can have any number of successor activities. The more there are and the more conditional ones there are, the higher the complexity of the process.
Use Case Complexity
Use cases as coined by Ivar Jacobson are instances of system usage 9. A user or system actor invokes a use case. This is a case of usage. The relationships between use cases may have different meanings. They can mean usage or extension or include or inherits. The more relations there are, the higher the usage complexity. The design goal is to reduce complexity by restricting the number of dependencies between use cases. On the other hand, if the application requires it then they have to be included. Otherwise the complexity is only pushed off to another layer.
Actor Interaction Complexity
System actors trigger the use cases. Any one actor can start one or more use cases. The more use cases there are per actor, the more complex is the relation between actors and the system. From the viewpoint of an actor, a system is complex if he has to deal with many use cases. The more use cases there are per actor, the higher the complexity. A system which has only one use case per actor is simple because it is partitioned in accordance with the actors. The design goal is to restrict the number of use cases per actor. Of course by having more actors, the size of the system in use case points increases.
Overall Design Complexity
The overall design complexity is computed as the relation between the sum of all design entities and the sum of all design relationships.
A design in which each entity has only a few relationships can be considered less complex than a system design in which the number of relationships per entity is high. This reflects complexity as the relation of the number of relationships between elements of a set and the number of elements in a set. The more elements there are the larger the size of the set. The more relationships there are, the higher the complexity of the set. The design goal is to minimize the number of relationships between design entities.
UML Design Quality Metrics
The design quality metrics are computations for calculating selected qualities. Quality is defined here as the relation of that state the model is in, relative to the state it should be in 10. Quality measurement presupposes a standard for the UML model. The actual state of the model is then compared with that standard. The closer the model is to fulfilling that standard, the higher is its quality. In German the overall design quality can be simply expressed by the ratio:
The upper bound of the metric is 1. If the IST exceeds the SOLL then the quality goal has been surpassed. A quotient coefficient of 0.5 indicates median quality. It should be remembered that quality is relative. By itself the ratio may not mean so much 11. However in comparison with the ratio derived from another designs in exactly the same way, it indicates that the one design has a better or lower quality than the other, at least in respect to the quality characteristic measured. Since there is no absolute quality scale, the quality of a system design can only be assessed in relation to the quality of another 12. For assessing the quality of a UML model the following quality characteristics were selected.
Degree of Class Coupling
Class Coupling is the inverse of Interaction Complexity. It is computed by the equation:
The more interactions and associations there are between objects and classes the greater the dependency of those objects and classes upon one another. This mutual dependency is referred to as coupling. Classes with a high coupling have a greater impact domain. If they are changed there is a greater chance that the other classes will also be affected. The design goal is to have as few dependencies as possible, i.e. the coupling should be low. This quality characteristic is founded on empirical evidence that high coupling is associated with a greater impact domain, with a higher error rate and with more maintenance effort [^Gymo09].
Degree of Class Cohesion
Class Cohesion is measured in terms of the number of data attributes in a class relative to the number of class methods. It is computed by the equation:
The notion of cohesion denotes the degree to which the functions of a module belong together 13. Functions belong together when they process the same data. This can be referred to as data coupling. Thus, the fewer the data used by the same functions the better. Classes with a high cohesion have many methods and few attributes. Classes with many attributes and few methods have a lower cohesion. The design goal is to have as few common attributes for the same methods as possible. This quality characteristic is founded on the hypothesis that high cohesion is associated with high maintainability. This hypothesis has never really been proven.
Degree of Modularity
Modularity is a measure of decomposition. It expresses the degree to which a large system has been decomposed into many small pieces. The theory is that it is easier to deal with smaller units of code 14. The modularity of classes is determined by the number of attributes and methods a class has. It is expressed by the equation:
There is a prevailing belief undermined by numerous field experiments that many smaller units of code are easier to change than fewer larger ones. The old roman principle of “divide et imperum” applies to software as well. It has not been proven that smaller modules will necessarily be more error free. Therefore, the justification for modularity is based on the ease of change. In measuring code, modularity can be determined by comparing the actual size of the code units in statements to some predefined maximum size. In an object-oriented design, the elementary units are the methods. The number of methods per class should not exceed a defined limit. In measuring the modularity of UML it is recommended here to compare the total number of methods with the minimum number of methods per class multiplied by the total number of classes. The design goal here is to have as few methods as possible per class so as to encourage the designer to create more and smaller classes.
Degree of Portability
Portability at the design level is a measure of the ease with which the architecture can be ported to another environment. It is influenced by the way the design is packaged. Many small packages can be more easily ported then a few large ones. Therefore it is important to keep the size of the packages as small as possible. The package size is a question of the number of classes per package. At the same time packages should have only few dependencies on their environment. The fewer interfaces each package has, the better. The portability of a system is expressed in the equation:
The justification of this quality attribute goes along the same line as that of modularity. The number of classes per package should not exceed a given limit, nor should a package have more than a given number of interfaces with its environment, since interfaces bind a package with its environment. The design goal is to create packages with a minimum number of classes and interfaces.
Degree of Reusability
Reusability is a measure of the ease with which code units or design units can be taken out of their original environment and transplanted to another environment. That means there should be a minimum of dependency between design units 15. Dependencies are expressed in UML as generalizations, associations and interactions. Therefore, the equation for measuring the degree of dependency is:
The more generalizations, associations and interactions there are, the more difficult it is to take out individual classes and methods from the current architecture and to reuse them in another. As with plants, if their roots are entangled with the roots of neighboring plants, it is difficult to transplant them. The entangled roots have to be severed. This applies to software as well. The degree of dependency should be as low as possible. Inheritance and interaction with other classes raises the level of dependency and lowers the degree of reusability. The design goal here is to have as few dependencies as possible.
Degree of Testability
Testability is a measure of the effort required to test a system relative to the size of that system 16. The less effort is required, the higher the degree of testability. Test effort is driven by the number of test cases required to test as well as by the width of the interfaces, whereby that width is expressed as the number of parameters per interface. The equation for computing testability is:
The number of test cases required is computed based on the number of possible paths thru the system architecture. To test an interface the parameters of that interface have to be set to different combinations. The more parameters it contains, the more combinations have to be tested. Field experience has proven that it is easier to test several narrow interfaces, i.e. interfaces with few parameters, than to test a few wide interfaces, i.e. interfaces with many parameters. Thus, not only the number of test cases but also the width of the interfaces affects the test effort. The design goal here is to design an architecture which can be tested with the least possible effort. This can be achieved by minimizing the possible paths through the system and by modularizing the interfaces.
Degree of Conformity
Conformity is a measure of the extent to which design rules are adhered to. Every software project should have a convention for naming entities. There should be prescribed names for data attributes and interfaces as well as for classes and methods. It is the responsibility of the project management to see that these naming conventions are made available. It is the responsibility of the quality assurance to ensure that they are adhered to. The equation for conformity is very simple:
Incomprehensible names are the greatest barrier to code comprehension. No matter how well the code is structured it will remain incomprehensible as long as the code content is blurred by inadequate data and procedure names. The names assigned in the UML diagrams will be carried over into the code. Therefore, they should be selected with great care and conform to a rigid naming convention. The design goal here is to get the designers to use meaningful, standardized names in their design documentation.
Degree of Consistency
Consistency in design implies that the design documents agree with one another. One should not refer to a class or method in a sequence diagram, which is not also contained within a class diagram. To do so is to be inconsistent. The same applies to the methods in the activity diagrams. They should correspond to the methods in the sequence and class diagrams. The parameters passed in the sequence diagrams should also be the parameters assigned to the methods in the class diagrams. Thus, the class diagrams are the base diagrams. All of the other diagrams should agree with them. If not, there is a consistency problem. The equation for computing consistency is:
When measuring the degree of consistency, we encounter one of the greatest weaknesses of the UML design language. It is in itself inconsistent. That is because it was pasted together out of many different design diagram types, each with its own origin. State diagrams, activity diagrams and collaboration diagrams existed long before UML was born. They were taken over from structured design. The basis of the object-oriented design is the class diagram from Grady Booch 17. Use case and sequence diagrams were added later by Ivar Jacobson. So there was never a consistent design of the UML language. The designer has the possibility of creating the diverse diagram types totally independent of one another. If the UML design tool does not check this, it leads to inconsistent naming. The design goal here is to force the designers to use a common name space for all diagrams and to ensure that the methods, parameters and attributes referenced are defined in the class diagrams.
Degree of Completeness
Completeness of a design could mean that all of the requirements and use cases specified in the requirement document are covered by the design documentation. To check that would require a link with the requirement repository and to require that the same names are used for the same entities in the design as are used in the requirement text. Unfortunately the state of information technology is far removed from this ideal. Hardly any IT projects have a common name space for all of their documents let alone a common repository. Therefore, what is measured here is only formal completeness, i.e. that all of the diagrams required are also present. Degree of completeness is a simple relation of finished documents to required documents.
The design goal here is to ensure that all UML diagram types required for the project are actually available. As witnessed all of the UML projects ever tested by this author, the design is never completed. The pressure to start coding is too great and once the coding is started the design becomes obsolete.
Degree of Compliance
The ultimate quality of a system design is that it fulfills the requirements. Not everything which is measured is important and much of what is important is not measurable 18. That certainly applies here. Whether or not the user requirements are really fulfilled, can only be determined by testing the final product against the requirements. The most that can be done here is to compare the actors and use cases in the design with those specified in the requirements. Every functional requirement should be assigned to a use case in the requirement document. Assuming this to be the case the use cases in the requirement document should cover all functional requirements. If the number of use cases in the design matches the number of use cases in the requirements, we can consider the design to be compliant with the requirements, at least formally. This can be expressed in the coefficient:
If there are more use cases designed than were required, this only shows that the solution is greater than the problem. If there are less use cases in the design, then the design is obviously not compliant. The design goal here is to design a system which covers all requirements, at least at the use case level.
UML Design Size Metrics
The design size metrics are computed values for representing the size of a system. Of course what is being measured here is not the system itself, but a model of the system. The system itself will only be measurable when it is finished. One needs size measures at an early stage in order to predict the effort that will be required to produce and test a system. Those size measures can be derived from the requirements by analyzing the requirement texts or at design time by analyzing the design diagrams. Both measurements can, of course, be only as good as the requirements and/or the design being measured. Since the design is more detailed and more likely to be complete, the design size metrics will lead to a more reliable estimate. However, the design is complete much later than the requirements. That means the original cost estimation has to be based on the requirements. If the design based estimation surpasses the original one, it will be necessary to delete functionality, i.e. to leave out less important use cases and objects. If the design based estimation varies significantly from the original one, it will be necessary to stop the project and to renegotiate the proposed time and costs. In any case the project should be recalculated when the design is finished.
There are several methods for estimating software project costs 19. Each is based on a different size metric. When estimating a project one should always estimate with at least three different methods. For that reason five measures are taken to give the estimator a choice. The five size measurements taken are:
- Use Case-Points
Data-Points is a size measure originally published by Sneed in 1990 20. It is intended to measure the size of a system based solely on its data model but including the user interfaces. It is a product of the 4th Generation software development where the applications are built around the existing data model. The data model in UML is expressed in the class diagrams. The user interfaces may be identified in the use case diagrams. This leads to the following computation of data-points:
Function-Points is a size measure originally introduced by Albrecht at IBM in 1979 21. It is intended to measure the size of a system based on its inputs and outputs together with its data files and interfaces. Inputs are weighted from 3 to 6, outputs from 4 to 7, files from 7 to 15 and system interfaces from 5 to 10. This method of system sizing was based on the structured systems analysis and design technique. It has evolved over the years but the basic counting scheme has remained unchanged [^GaHe00]. It was never intended for object-oriented systems but it can be adapted. In a UML design, the closest to the logical files are the classes. The closest to the user inputs and outputs are the actor/usecase interactions. The interfaces between classes can be interpreted as system interfaces. With that rough approximation we come to the following computation of function-points:
Object-Points were designed specifically for measuring the size of object-oriented systems by Sneed in 1996 19. The idea was to find a size measure which could readily be taken from an object-design. As such it fits perfectly to the UML design. Object-Points are obviously the best size measure of an object model. Classes weigh 4 points, methods weigh 3 points, interfaces weigh 2 points and attributes/parameters weigh one point. That way, object-points are computed as:
UC-Points were introduced by a Swedish student working at Ericsson by the name of G. Karner in 1993 22. The idea here was to estimate the size of a software system based on the number of actors and the number and complexity of the use cases. Both actors and use cases were classified in three levels – simple, medium and difficult. Actors are rated on a scale of 1 to 3. Use Cases are rated now on a scale of 5 to 15 23. The two are multiplied together to give the unadjusted use-case points. This method is also appropriate for measuring the size of a UML design provided the use cases and actors are all specified. Here the median levels are used to classify all actors and use cases, but are enhanced by the number of actor to use case interactions.
Test-Cases were first used as a size measure by Sneed in 1978 for estimating the effort to test the Siemens Integrated Transport System – ITS. The motivation behind it was to charge the module test on the basis of tested cases. A test case was defined to be equivalent to a path through the test object. Much later the method was revitalized to estimate the costs of testing systems 24. In testing systems a test cases is equivalent to a path through the system. It begins at the interaction between an actor and the system and either follows a path through the activity diagrams or transcends the sequence diagrams via interactions between the classes. There should be one test case for every path through the interaction diagrams as well as for every object state specified in the state diagrams. Thus, the number of test cases can be derived from the use case interactions times the number of class interactions times the number of object states. It is computed as follows:
Automated Analysis of UML Designs with UMLAudit
To measure UML-designs the tool UMLAudit was developed. UMLAudit is a member of the SoftAudit automated quality assurance tool set. This tool set also includes analysis tools for English and German requirements texts as well as for all of the leading programming languages, the most popular database schema languages and several user and system interface definition languages. UMLAudit contains an XML Parser which parses the XML files produced by the UML modeling tool to represent the diagrams 25. There the diagram and model entity types, names and relationships are included as attributes, which are readily recognized by their model types and names. The object of measurement is the XML schema of the UML-2 model with its model types as specified by the OMG 26
The first step of UMLAudit is to collect the design types and names from the XML files and store them in tables. The second step is to go through the tables and count them. The third step is to check the names against the name convention templates. The final step is to check the referential consistency by comparing the entities referenced with the entities defined. As a result two outputs are produced:
- a UML deficiency report and
- a UML metric report.
The UML Deficiency Report is a log of rule violations and discrepancies listed by diagram. At present there are only two types of deficiencies:
- Inconsistent references and
- Name rule violations.
If a diagram such as a state, activity or sequence diagram references a class, method or parameter not defined in the class diagram, an inconsistent reference is reported. If the name of an entity deviates from the naming rules for that entity type, a naming violation is reported. These deficiencies are summed up and compared with the number of model types and type names to give the design conformance.
The UML Metric Report lists out the quantity, complexity and quality metrics at the file and system level. The quantity metrics are further subdivided into diagram quantities, structural quantities, relationship quantities and size metrics.
Judging a software system by its design is like judging a book by its table of contents. If the table is very fine grained you will be able to assess the structure and composition of the book and to make assumptions about the content. The same is the case for UML. If the UML design is fine grained down to a detailed level, it is possible to make an assessment and to estimate costs based on the design 27. If it is only coarse grained, the assessment of the system will be superficial and the estimation unreliable. Measuring size, complexity and quality of anything can only be as accurate as the thing one is measuring. UML is only a model and models do not necessarily reflect reality 28. In fact they seldom do. UML models are in practice often incomplete and inconsistent, making it difficult to base a test upon them. This remains the biggest obstacle to model-based testing.
- Briand, L./Morasca,S./Basili,V.: “Property-based Software Engineering Measurement”, IEE Trans. On S.E. Vol. 22, No. 1, Jan. 1996, p. 68 ↩
- Briand, L./Morasca,S./Basili,V.: “An Operational Process for Goal-Driven Definition of Measures”, IEEE Trans. On S.E. Vol. 28, No. 12, Dec. 2002, p. 1106 ↩
- Hausen, H.-L./Müllerburg, M.: “Über das Prüfen, Messen und Bewerten von Software“, Informatik Spektrum, No. 10, 1987, p. 123 ↩
- McCabe, T./Butler, C.: “Design Complexity Measurement and Testing”, Comm. Of ACM, Vol.32, No. 12, Dec. 1989, p. 1415 ↩
- Booch, G.: “Measuring Architectural Complexity”, IEEE Software, July 2008, p. 14 ↩
- Chidamer, S./Kemerer, C.: „“A Metrics Suite for object-oriented Design”, IEEE Trans on S.E., Vol. 20, No. 6. 1994, p. 476 ↩
- Harrison,R./Counsel,S./Reuben, V.: “An Evaluation of the MOOD Set of Object-oriented Software Metrics”, IEEE Trans. On S.E., Vol. 24, No. 6, June 1998, p. 491 ↩
- McCabe, T.: “A Complexity Measure”, IEEE Trans S.E., Vol. 2, No. 6, 1976, p.308 ↩
- Jacobson, I., a.o: Object-Oriented Software Engineering – A Use Case driven Approach, Addison-Wesley Pub., Wokingham, G.B., 1993, p. 153 ↩
- Card, D./Glass,R.: Measuring Software Design Quality, Prentice-Hall, Englewood Cliffs, 1990, p. 42 ↩
- Erdogmus, H.: “The infamous Ratio Measure”, IEEE Software, May 2008, p. 4 [GaHe00] Garmus, D.; Herron, D.: Function-Point Analysis: Measurement Process for successful Software Projects, Addison-Wesley, Reading MA., December 15, 2000. ↩
- Rombach, H.-D.; “Design Measurement – Lessons learned”, IEEE Software, March 1990, p. 17 ↩
- Bieman, J./Ott, L.: “Measuring Functional Cohesion”, IEEE Trans on S.E., Vol. 20, No. 8, August 1994, p. 644 ↩
- Sarkar,S./Rama,G./Kak,A.: “Information-theoretic Metrics for measuring the Quality of Software Modularization”, Vol. 33, No. 1, Jan. 2007, p. 14 ↩
- Sneed, H.M.: „Metriken für die Wiederverwendbarkeit von Softwaresystemen“, in Informatikspektrum, Vol. 6, S. 18-20 (1997) ↩
- Sneed, H./ Jungmayr, S.: „Produkt- und Prozessmetriken für den Softwaretest“, Informatikspektrum, Band 29, Nr. 1, p. 23, (2006) ↩
- Booch, G.: “Object-oriented Development” IEEE Trans. On S.E., Vol. 12, No. 2, March 1986, p. 211 ↩
- Ebert, C./Dumke, R.: “Software Measurement, Springer Verlag, Berlin, 2007, p. 1 ↩
- Sneed, H.: “Schätzung der Entwicklungskosten von objektorientierter Software“, Informatikspektrum, Band 19, No. 3, June 1996, p. 133 ↩
- Sneed, H.: “Die Data-Point Methode“, Online, Zeitschrift für Datenverarbeitung, No. 5, May 1990, p. 48 ↩
- Albrecht, A.: “Measuring Application Development Productivity”, Proc of Joint SHARE, GUIDE and IBM Symposium, Philadephia, Oct. 1979, p. 83 ↩
- Karner, G.: Metrics for Objectory, Diplomarbeit, University of Linköping, Sweden, Nr. Lith-IDA-Ex-9344:21, Dec. 1993 ↩
- Ribu, K.: Estimating Object-Oriented Projects with Use Cases, Masters Thesis, University of Oslo, Norway, November 2001 ↩
- Sneed, H./Baumgartner,M./Seidl,R.: Der Systemtest, Hanser Verlag, München/Wien, 2008, p. 59 ↩
- Bock, C.: “UML without Pictures”, IEEE Software, Sept. 2003, p. 35 ↩
- Object Management Group, “UML 2 XML Schema, Version 2.1, April 2003, www.omg.org/cgi-bin/doc?ad/03-04-02 ↩
- Miller, J.: “What UML should be”, Comm. Of ACM, Vol. 45, No. 11, Nov. 2002, p. 67 ↩
- Selic, B.: “The Pragmatics of Model-Driven Development”, IEEE Software, Sept. 2003, p. 19 ↩