What Are UML Metrics?
UML metrics are measures derived from a UML model and its diagrams. They count a model’s diagrams across the UML diagram types, plus the entities and relationships those diagrams contain. These counts make the size, complexity, and quality of a software design measurable long before the system itself is finished. The same counts also give an estimate of how much development and test effort a project is going to take.
The metrics fall into four groups. Quantity metrics count diagrams, entities, and relationships. Complexity metrics relate entities to relationships. Quality metrics compare the actual state of the model with a target state. Design size metrics derive size measures from the model for effort estimation. I wrote this text together with Harry Sneed; it goes back to the talk “Design Metrics for UML Models” at IWSM Mensura 2010 and keeps the 2010 count of nine diagram types, while the current UML 2.5.1 distinguishes 14.
What Types of UML Diagrams Are There?
The UML specification of the Object Management Group (OMG), version 2.5.1 from December 2017, sorts 14 UML diagram types into two groups (Annex A, Figure A.5). Structure diagrams show the static structure of a system, everything that holds regardless of time. Behavior diagrams show the dynamic behavior of its objects, including their methods, collaborations, activities, and state histories. Each group contains seven diagram types:
| Structure Diagrams | Behavior Diagrams |
|---|---|
| Class diagram | Use case diagram |
| Object diagram | Activity diagram |
| Package diagram | State machine diagram |
| Component diagram | Sequence diagram |
| Composite structure diagram | Communication diagram |
| Deployment diagram | Interaction overview diagram |
| Profile diagram | Timing diagram |
Within the behavior diagrams, sequence, communication, interaction overview, and timing diagrams form the subgroup of interaction diagrams. The boundaries aren’t rigid, though. The specification explicitly allows mixing structure and behavior in one diagram, such as a state machine nested inside an internal structure.
For measurement, the difference between model and diagram matters. A UML model consists of elements such as packages, classes, and associations. A diagram shows part of that model graphically. UML metrics count both: the diagrams and the model elements that appear in them.
Quantity Metrics: What Gets Counted in a UML Model
Quantity metrics are simple counts of the diagram and model types in a UML model. The model types are further split into entities and relationships.
The metrics count the number of each of these nine diagram types:
- Use case diagrams
- Activity diagrams
- Class diagrams
- Sequence diagrams
- Interaction diagrams
- State diagrams
- Component diagrams
- Deployment diagrams
- Design diagrams
They also count these 17 entity types:
- Subsystems
- Use cases
- Actors
- Components
- Interfaces
- Classes
- Base classes and superclasses
- Methods
- Parameters
- Attributes
- Activities
- Objects
- States
- Rules
- Stereotypes
- Design entities
- Referenced design entities
And these ten relationship types:
- Usages
- Associations
- Generalizations
- Interactions
- Class hierarchy levels
- Method calls
- Activity flows
- State transitions
- Test cases
- Design relationships
These counts were selected because they relate to the goals of object-oriented system design. The selection was based on Basili and Rombach’s goal-question-metric method.
UML Complexity Metrics: How Complex Is a Design?
Complexity metrics calculate selected kinds of complexity. Complexity is defined here as the ratio of entities to relationships. The size of a set is the number of its elements, and its complexity is the number of relationships between those elements. The more connections or dependencies there are relative to the number of elements, the higher the complexity. The same applies to a single entity: its complexity comes from the number of its sub-entities relative to the relationships between them. That makes the overall complexity of a design easy to express:
This leads to the following complexity types for UML models.
Object Interaction Complexity
The more interactions there are between objects and the more associations between classes, the higher the complexity. This takes in both the abstract level of classes and the physical level of objects. The measure is an inverse coupling metric. It rests on the empirical finding that systems with many dependencies between their parts are hard to maintain.
Class Hierarchy Complexity
The more levels a class hierarchy has, the more the classes at the lower levels depend on those above them. Deep inheritance has long been criticized for driving up complexity. This metric corresponds to Chidamber and Kemerer’s depth of inheritance tree (DIT) metric. It is backed by empirical evidence that object-oriented systems with deep inheritance trees (more than three levels, for example) are more error-prone than others.
Class Data Complexity
The more data attributes a class has, the higher its complexity. The design goal is many classes with few data attributes each rather than few classes with many attributes. The assumption behind it is that smaller sets of data are easier to test and maintain.
Class Function Complexity
The more methods, or functions, a class has, the higher its complexity. The metric assumes that every class has at least two implicit functions: a constructor and a destructor. This corresponds to Chidamber and Kemerer’s weighted methods per class (WMC) metric with every method weighted as one. The design goal is many classes with as few functions as possible rather than few classes with many methods. The assumption: a system whose functionality is split into many small pieces is easier to maintain and test.
Object State Complexity
Objects are instances of a class, and objects have states. The more states they have, the more complex they are. A simple class is a singleton with one object in one static state. A complex class has several objects, each with several possible states. Neither the CK nor the MOOD metrics take state complexity into account, even though, together with the cyclomatic complexity of the methods, it is one of the main drivers of test effort. The design goal is as few object states as possible, although the application ultimately decides how many there are. If an object such as an account has many states, say opened, balanced, overdrawn, locked or closed, every one of them has to be created and tested.
State Transition Complexity
The lines connecting the states in a state diagram represent transitions from one state to another. A state can have any number of successor states. The more it has, the more complex the state transition graph. As with McCabe’s cyclomatic complexity, the number is based on the edges and nodes of a graph; McCabe takes the difference, this metric the ratio. Here, though, the nodes are states rather than statements, and the edges are transitions rather than branches. The design goal is as few transitions as possible, because every state transition has to be tested at least once, and that drives up test costs.
Activity Control Flow Complexity
The lines connecting the activities in an activity diagram represent the control flow from one activity to the next. These flows can be conditional or unconditional, and conditional flows add to the complexity of the modeled process. An activity can have any number of follow-on activities. The more there are, and the more of them are conditional, the more complex the process.
Use Case Complexity
Use cases, as Ivar Jacobson defined them, are instances of system usage. A user or a system actor invokes a use case. Relationships between use cases can mean different things: usage, extension, inclusion or inheritance. The more relationships there are, the more complex the usage. The design goal is to limit the dependencies between use cases. If the application needs them, though, they have to be modeled. Otherwise the complexity just moves to another layer.
Actor Interaction Complexity
System actors trigger the use cases, each actor one or more of them. The more use cases each actor has, the more complex the relationship between actors and system. From an actor’s point of view, a system is complex when that actor has to deal with many use cases. A system with only one use case per actor is simple, because it is partitioned along its actors. The design goal is to limit the number of use cases per actor. More actors, of course, increase the size of the system in use case points.
Overall Design Complexity
The overall design complexity is the ratio between the sum of all design entities and the sum of all design relationships.
A design in which each entity has only a few relationships counts as less complex than one with many relationships per entity. This mirrors the definition of complexity via the ratio between the number of elements in a set and the relationships between them. More elements make the set larger, more relationships make it more complex. The design goal is to keep the relationships between design objects to a minimum.
UML Quality Metrics: How Do You Measure the Quality of a UML Model?
The design quality metrics calculate selected qualities. Quality is defined here as the ratio between the state the model is in and the state it should be in. Measuring it therefore requires a standard for the UML model against which the actual state is compared. The closer the model comes to that standard, the higher its quality. In simplified form, overall design quality can be expressed as a ratio:
The upper limit of the metric is 1. If the actual value exceeds the target value, the quality goal has been surpassed. A quotient of 0.5 indicates average quality. Keep in mind that quality is relative. On its own, the quotient says little. Compared with the quotient of another design, calculated in exactly the same way, it shows which design is better or worse with respect to the quality characteristic being measured. Because there is no absolute quality scale, the quality of a system design can only be judged relative to another design. The following quality characteristics were selected for UML models.
Degree of Class Coupling
Class coupling is the inverse of interaction complexity. It is calculated as follows:
The more interactions and associations there are between objects and classes, the more they depend on each other. This mutual dependency is called coupling. Tightly coupled classes have a wider reach: when they change, other classes are more likely to be affected. The design goal is as few dependencies as possible, in other words low coupling. The characteristic is based on the empirical finding that high coupling goes hand in hand with a larger impact area, a higher defect rate and more maintenance effort.
Degree of Class Cohesion
Class cohesion is measured by the number of data attributes in a class relative to the number of its methods:
Cohesion describes how closely the functions of a module belong together. Functions belong together when they process the same data, which you could call data coupling. The less data the same functions use, the better. Classes with high cohesion have many methods and few attributes, classes with many attributes and few methods have lower cohesion. The design goal is as few shared attributes as possible for the same methods. The characteristic rests on the hypothesis that high cohesion goes hand in hand with high maintainability. That hypothesis has never been proven.
Degree of Modularity
Modularity is a measure of decomposition. It expresses how far a large system has been broken down into many small parts. The theory behind it: smaller units of code are easier to handle. The modularity of classes follows from the number of their attributes and methods:
It is widely believed that many small units of code are easier to change than a few large ones, and numerous field trials support this. The old Roman principle of “divide et impera” applies to software, too. What has not been proven is that smaller modules necessarily contain fewer defects. The case for modularity therefore rests on ease of change. In code, you can determine modularity by comparing the actual size of code units in statements with a predefined maximum size. In an object-oriented design, the elementary units are the methods, and the number of methods per class should not exceed a defined limit. To measure the modularity of a UML model, it makes sense to compare the total number of methods with the minimum number of methods per class multiplied by the total number of classes. The design goal is as few methods per class as possible, which encourages designers to build more, smaller classes.
Degree of Portability
At the design level, portability describes how easily the architecture can be moved to another environment. It depends on how the design is divided into packages. Many small packages are easier to port than a few large ones, so packages should stay as small as possible. Package size is the number of classes per package. At the same time, packages should have few dependencies on their environment: the fewer interfaces a package has, the better. The portability of a system is expressed like this:
The reasoning is similar to that for modularity. The number of classes per package should not exceed a certain limit, and a package should have only a limited number of interfaces to its environment, because those interfaces are exactly what ties it to that environment. The design goal is packages with as few classes and interfaces as possible.
Degree of Reusability
Reusability describes how easily units of code or design can be taken out of their original environment and moved into another. That requires as few dependencies between design units as possible. In UML, dependencies are expressed as generalizations, associations and interactions. The degree of dependency is therefore calculated as follows:
The more generalizations, associations and interactions there are, the harder it is to detach individual classes and methods from the current architecture and reuse them elsewhere. Think of plants whose roots have grown into those of their neighbors: to move one, you have to cut through the roots. Software is no different. Inheritance and interaction with other classes raise the degree of dependency and lower reusability. The design goal is as few dependencies as possible.
Degree of Testability
Testability measures the effort needed to test a system relative to its size. The less effort required, the higher the testability. Test effort depends on the number of test cases and the width of the interfaces, measured as the number of parameters per interface. The equation is:
The number of test cases needed follows from the possible paths through the system architecture. To test an interface, its parameters have to be set in different combinations. The more parameters, the more combinations. Experience shows that several narrow interfaces with few parameters are easier to test than a few wide ones with many parameters. So besides the number of test cases, the width of the interfaces also drives test effort. The design goal is an architecture that can be tested with as little effort as possible. You get there by minimizing the possible paths through the system and modularizing the interfaces.
Degree of Conformity
Conformity measures how well design rules are followed. Every software project should have a naming convention for its entities, with prescribed names for data attributes and interfaces as well as for classes and methods. Project management is responsible for providing these naming conventions, quality assurance for making sure they are followed. The equation is very simple:
Unclear names are the biggest obstacle to understanding code. However well the code is structured, it stays hard to understand as long as data and procedure names are unclear. The names chosen in the UML diagrams end up in the code, so they should be chosen with care and follow a strict naming convention. The aim is for designers to use meaningful, standardized names in their design documentation.
Degree of Consistency
Consistency in a design means that the design documents match each other. A sequence diagram should not reference a class or method that is missing from the class diagram. The same goes for the methods in activity diagrams: they should match those in the sequence and class diagrams. And the parameters passed in sequence diagrams should be the same ones assigned to the methods in the class diagrams. The class diagrams are the foundation, and all other diagrams should fit them. If they don’t, there is a consistency problem. The equation is:
Measuring consistency exposes one of the biggest weaknesses of UML: the language is inconsistent in itself. That is because it was assembled from many types of design diagrams with different origins. State, activity and collaboration diagrams existed long before UML and come from structured design. The basis of object-oriented design is Grady Booch’s class diagram. Ivar Jacobson added use case and sequence diagrams later. So there was never a unified design for UML as a language. Designers can create the diagram types completely independently of each other. If the UML tool doesn’t check them, inconsistent names creep in. The design goal is therefore a shared namespace for all diagrams, in which every referenced method, parameter and attribute is defined in the class diagrams.
Degree of Completeness
Completeness could mean that the design documentation covers every requirement and use case in the requirements document. Checking that would require a link to the requirements repository, and the design would have to use the same names for the same entities as the requirements text. Practice is a long way from that. Hardly any IT project has a shared namespace for all its documents, let alone a shared repository. So only formal completeness is measured, meaning whether all required diagrams exist. The degree of completeness is a simple ratio of finished to required documents:
The design goal is that every UML diagram type the project needs actually exists. Testing UML projects shows the same thing again and again: the design is never finished. The pressure to start coding is too high, and once coding starts, the design goes out of date.
Degree of Compliance
What matters most about a system design is that it meets the requirements. Not everything that gets measured is important, and much of what is important can’t be measured. That is certainly the case here. Whether the users’ requirements have really been met only becomes clear when the finished product is tested against them. The most you can do with the design is compare its actors and use cases with those in the requirements. Every functional requirement should be assigned to a use case in the requirements document, so that the use cases cover all functional requirements. If the number of use cases in the design matches the number in the requirements, the design counts as compliant, at least formally. This coefficient expresses it:
If more use cases were designed than required, the solution is bigger than the problem. If fewer were designed, the design is clearly not compliant. The goal is a design that covers all requirements, at least at the level of use cases.
UML Design Size Metrics: Estimating Effort from the Design
Design size metrics are calculated values that represent the size of a system. What gets measured is the system’s model, because the system itself can only be measured once it is finished. Size measures are needed early, though, to predict development and test effort. They can be derived from the requirements by analyzing the requirements texts, or at design time from the design diagrams. Both measurements are only as good as the requirements or the design being measured. Since the design is more detailed and more likely to be complete, measuring design size gives the more reliable estimate. But the design is complete much later than the requirements, so the first cost estimate has to be based on the requirements. If the design-based estimate is higher than the original one, functionality has to go, meaning less important use cases and objects. If it differs significantly, the project has to be stopped and time and cost renegotiated. Either way, the project should be recalculated once the design is finished.
There are several methods for estimating software project costs, and each is based on a different size metric. A project should always be estimated with at least three different methods. That is why five size measures are available here:
- Data points
- Function points
- Object points
- Use case points
- Test cases
Data Points
Data points is a size measure Sneed published in 1990. It measures the size of a system based solely on its data model, but includes the user interfaces. It comes from fourth-generation software development, where applications are built around an existing data model. In UML, the data model lives in the class diagrams, and the user interfaces can be identified in the use case diagrams. That gives the following calculation of data points:
Function Points
Function points is a size measure Albrecht introduced at IBM in 1979. It measures the size of a system by its inputs and outputs along with its data files and interfaces. Inputs are weighted 3 to 6, outputs 4 to 7, files 7 to 15 and system interfaces 5 to 10. The method is based on structured systems analysis and structured design. It has evolved over the years, but the basic counting scheme has stayed the same. It was never meant for object-oriented systems, but it can be adapted. In a UML design, classes come closest to logical files, and the interactions between actors and use cases come closest to user inputs and outputs. The interfaces between classes can be read as system interfaces. With this rough approximation, function points are calculated as follows:
Object Points
Sneed developed object points in 1996 specifically to measure the size of object-oriented systems. The idea was a size measure that could easily be derived from an object design. That makes it a very good fit for UML designs, and for an object model it is the obvious size measure. Classes count 4 points, methods 3 points, interfaces 2 points and attributes or parameters one point each. Object points are calculated like this:
Use Case Points
Use case points were introduced in 1993 by Gustav Karner at the Swedish company Objectory AB. The idea: estimate the size of a software system from the number of actors and the number and complexity of use cases. Actors and use cases are each classified at three levels: simple, medium and complex. Actors are rated on a scale of 1 to 3, use cases on a scale of 5 to 15. Adding the two gives the unadjusted use case points. The method also works for measuring the size of a UML design, provided all use cases and actors are specified. Here, median values are used to classify all actors and use cases, extended by the number of interactions between actors and use cases.
Test Cases
Sneed first used test cases as a size measure in 1978 to estimate the test effort for the Siemens Integrated Transport System (ITS). The aim was to calculate the module test based on test cases, with one test case corresponding to one path through the test object. Much later, the method was revived to estimate the cost of system testing. In a system test, a test case corresponds to a path through the system. It starts with an interaction between an actor and the system and either follows a path through the activity diagrams or runs through the sequence diagrams via the interactions between classes. There should be one test case for every path through the interaction diagrams and for every object state specified in the state diagrams. The number of test cases is therefore the number of use case interactions times the number of class interactions times the number of object states:
Automated Analysis of UML Designs with UMLAudit
The UMLAudit tool was developed to measure UML designs. It was part of the SoftAudit toolset for automated quality assurance, which also included analyzers for English and German requirements texts, for all leading programming languages, for the most common database schema languages and for several languages used to define user and system interfaces. UMLAudit had an XML parser that read the XML files the UML modeling tool used to display the diagrams. The types, names and relationships of the diagrams and model instances were captured as attributes that are easy to identify by their model types and names. The object of measurement was the XML schema of the UML 2 model with its model types as specified by the OMG.
UMLAudit worked in four steps. First, it collected the design types and names from the XML files and stored them in tables. Next, it went through the tables and counted. In the third step, it checked the names against the naming convention templates. Finally, it checked referential consistency by comparing the referenced entities with the defined ones. The result was two reports:
- a UML defect report and
- a UML metrics report.
The UML defect report was a log of rule violations and discrepancies, sorted by diagram. It recognized two kinds of defects:
- inconsistent references and
- naming rule violations.
If a diagram, such as a state, activity or sequence diagram, referred to a class, method or parameter that was not defined in the class diagram, UMLAudit reported an inconsistent reference. If an entity’s name departed from the naming rules for its entity type, it reported a naming violation. The defects were added up and compared with the number of model types and type names to determine design conformity.
The UML metrics report listed the quantity, complexity and quality metrics at file and system level. The quantity metrics were further divided into diagram quantities, structure quantities, relationship quantities and size metrics.
What a Design Reveals about the System
Judging a software system by its design is like judging a book by its table of contents. If the table of contents is very detailed, you can judge the structure and organization of the book and make assumptions about its content. UML works the same way. If the UML design is worked out down to a detailed level, you can assess the system and estimate its cost from the design. If it is only rough, the assessment stays superficial and the estimate unreliable.
You can only measure size, complexity and quality as precisely as the thing you are measuring allows. And UML is only a model. Models don’t necessarily reflect reality, and in fact they rarely do. In practice, UML models are often incomplete and inconsistent, which makes it hard to build a test on them. That remains the biggest obstacle to model-based testing.
Frequently Asked Questions
How can the complexity of a software design even be measured?
Complexity is expressed as the ratio of entities to relationships. The size of a set is determined by the number of its elements, whereas complexity is determined by the number of connections and dependencies between these elements. The more relationships an element has, the higher the value. Accordingly, the design goal is to keep the number of relationships between design objects as small as possible.
Are deep inheritance hierarchies a problem in object-oriented design?
Yes. The more levels a class hierarchy has, the more the lower-level classes depend on the higher-level ones. There is empirical evidence that object-oriented systems with deep inheritance trees (for example, more than three levels) are more prone to errors than others. This measure corresponds to the depth of inheritance tree (DIT) metric from the CK metrics.
Is the benefit of high class cohesion scientifically proven?
No. The hypothesis that high cohesion leads to high maintainability has never actually been proven. Cohesion is measured by the ratio of data attributes to methods in a class: classes with many methods and few attributes are considered cohesive, while classes with many attributes and few methods are considered weakly cohesive. The assumption that smaller modules are less error-prone is similarly poorly substantiated.
How can the future testing effort be estimated during the design phase?
Two factors determine it: the number of required test cases and the breadth of the interfaces, expressed by the number of parameters per interface. The number of test cases is derived from the possible paths through the architecture. Practical experience shows that multiple narrow interfaces are easier to test than a few broad ones with many parameter combinations.
Why do the individual diagrams in a UML model often not fit together?
Because UML itself is internally inconsistent. It was assembled from diagram types of different origins: state, activity, and collaboration diagrams come from structured design; the class diagram from Grady Booch; and use-case and sequence diagrams were added later by Ivar Jacobson. The designer can create all diagram types independently of one another. Class diagrams should serve as the foundation.
Should project effort be estimated based on the requirements or on the design?
Design-based estimation is more reliable because the design is more detailed and likely more complete. However, it comes too late for the initial calculation, which must therefore be based on the requirements. If the later estimate exceeds the original one, functionality must be omitted, such as less important use cases. In the event of a significant deviation, the project should be halted and renegotiated.
How meaningful are metrics derived from a UML model?
They are only as accurate as the model itself. To illustrate: Evaluating a system based on its design is like judging a book by its table of contents. If the design is detailed, evaluation and cost estimation are possible; if it is rough, both remain superficial. In practice, the models were often incomplete and inconsistent, which made model-based testing difficult.
