Metrics Derivation

This section defines and describes in detail the necessary statistics to be collected, the metrics derived from these statistics, and some general guidelines for their interpretation. Appendix D provides detailed examples of a real-world application to illustrate further how such metrics can be used to manage and control a project. The derivations are not an obvious top-down progression; rather, they resulted from substantial trial and error, numerous empirical analyses, intuition, and heuristics.

The raw statistics to be collected include numbers and types of software changes, SLOC damaged, and SLOC fixed. The challenges are to find the right filtering techniques for the raw rework statistics that identify useful trends, and to uncover objective measures to quantify progress (intermediate attributes during development) and quality (attributes of the end product). The final objective is to quantify the product's modularity, adaptability, maturity, and maintainability. Modularity and adaptability are intuitively easy to define as a function of rework; maturity and maintainability are more subtle.

• Modularity. This metric measures the average extent of breakage or scrap. It identifies the need to quantify the scrap (volume of SLOC damaged) and the number of instances of rework (number of SCOs). In effect, modularity is defined as a measure of breakage localization, with a lower value being better.

• Adaptability. This metric measures the average complexity of breakage as measured in rework. It identifies the need to quantify the rework (effort required for resolution) and the number of instances of rework (number of

SCOs). Adaptability quantifies the ease of change, with a lower value being better.

• Maturity. Intuitively, maturity corresponds to the level of trustworthiness of the product. Objectively, this metric measures the defect rate. The goal is to be defect-free—namely, to achieve infinite maturity. The trust increases primarily through extended usage. Because software is intellectual property, not comprised of physical matter, it does not wear out. Software should mature over time, meaning that its users (test team, beta users, users of a released product) should experience defects less frequently with each subsequent release of the product. This statement assumes constant functionality and performance in new releases. The expectation of increasing maturity in new releases is valid even when functionality and performance change. Similarly, there should be noticeable trends in release maturity across the development life cycle of a healthy iterative development effort. A simple indicator of the defect rate would require measuring the number of defects (type 0 and 1 SCOs) and the amount of usage time. From these parameters, the mean time between failures (MTBF) can be derived for a given release. Higher vales of maturity are better, reflecting the average time between defects perceived by a user.

• Maintainability. Theoretically, the maintainability of a product is related to the productivity with which the maintenance team can operate. Productivity is so difficult to compare among projects, however, that this definition is intuitively unsatisfying. The ratio of the productivity of rework to the productivity of development provides a value that is independent of productivity yet reflects the development complexity. The ratio normalizes the project productivity differences and provides a relatively comparable metric. Consequently, maintainability is defined as the ratio of rework productivity to development productivity. Intuitively, this value identifies a product that can be changed three times as efficiently as it was developed (maintainability = 0.33) as having better (lower) maintainability than a product that can be changed twice as efficiently (maintainability = 0.5) as it was developed, independent of the absolute maintenance productivity realized. The statistics needed to compute these values are the total development effort, total SLOC, total rework effort, and total reworked SLOC.

While these values provide useful objective measures of end products, their intermediate values as a function of time also provide insight during development into the expected end-product values. Once a project has gained some experience with maintenance of early increments, this experience should be useful for predicting the rework inherent in remaining increments.

This brief derivation was relatively simple. It is not necessary to deal with these metrics as a complete set, although multiple perspectives are needed by project management to achieve accuracy. Subsets, or different sets, are also useful. Most of the analysis, mathematics, and data collection inherent in these metrics should be automated so that practitioners need only interpret the results and understand their basis.

C.2.1 Collected Statistics

Some specific statistics must be collected over the software project life cycle to implement the proposed metrics. These statistics, identified in Table C-l, include the following:

• Total SLOC (SLOCj). This statistic tracks the estimated total size of the product under development. This value may change significantly over the life of the development as early requirements unknowns are resolved and design solutions mature. This total should also include reused software, which is part of the delivered product and subject to change by the development team.

• Configured SLOC (SLOC^). This statistic tracks the transition of software components from a maturing design state into a controlled configuration. For any given project, this statistic will provide insight into progress and

Table C-l. Definitions of collected statistics

collected statistics


Total SLOC

SLOCy = total size in SLOC

Configured SLOC

SLOCc = current baseline SLOC

Critical defects

SCOq = number of type 0 SCOs

Normal defects

SCOj = number of type 1 SCOs


SCO2 = number of type 2 SCOs

New features

sco3 = number of type 3 SCOs

Number of SCOs

N = SCO0 + SCO, + SC02

Open rework (breakage)

B = cumulative broken SLOC due to SCOq, SCO], and SC02

Closed rework (fixes)

F = cumulative fixed SLOC

Rework effort

E = cumulative effort expended fixing SCOq, SCOj, and SCO2

Usage time

UT = hours that a given baseline has been operating under realistic

Project Management Made Easy

Project Management Made Easy

What you need to know about… Project Management Made Easy! Project management consists of more than just a large building project and can encompass small projects as well. No matter what the size of your project, you need to have some sort of project management. How you manage your project has everything to do with its outcome.

Get My Free Ebook

Post a comment