My approach to metrics is similar to that of DeMarco, who proposes to measure software quality through the absence of spoilage [DeMarco, 1982]. To remain technology- and project-independent, his definitions are purposely vague; mine are quite explicit. Consistency of application is important for accurate interpretation, just as it is with cost estimation techniques. Software cost estimation has subjective inputs and objective outputs. My approach will define objective inputs, which may require subjective interpretation within the context of a specific project.
One effective way to assess software quality over a life cycle is by measuring rework in the configured baselines. The unit of measurement can be source lines of code, function points, object points, files, components, or some other measure of the software size. This discussion uses SLOC as the primitive size metric because it is used predominantly by the industry, is the easiest measure to understand, and best matches the case study data in Appendix D.
In some cases, the software quality assessment derived from an objective collection of change metrics will require context-dependent assessment. Judgment is needed to assess quality using any metric. The same metrics should be used to assess quality during development (trend-oriented) and following development (value-oriented). For example, the volume of rework following product delivery is an objective measure of quality or lack of quality. The amount of rework following the first configuration baseline during development is ambiguous without further context. Zero rework might be interpreted as a perfect baseline (which is unlikely),-an inadequate test program, or an unambitious first build.
It is extremely difficult to make this concept objective. Only two mechanisms are available for defining customer expectations of quality: software requirements specifications for function and performance, and an approved expenditure plan that quantifies cost and schedule goals. These artifacts, which basically correspond to the contract, are traditionally the lowest quality products produced by a project because they must be agreed upon early in the life cycle, when there are still too many unknowns. A modern, iterative process and objective software metrics should provide better insight into the extent to which function, performance, cost, and schedule comply with customer expectations.
SCOs, discussed at length in Chapter 12, constitute direction to proceed with changing a configured software component. (SCOs are often called software problem reports, but problem has a negative connotation, and not all changes are motivated by problems.) The change may be needed to rework a poor-quality component (type 0 or 1, a fix), to rework a component to achieve better quality (type 2, an enhancement), or to accommodate a customer-directed change in requirements (type 3, a scope change). The difference between a fix and an enhancement is inherent in the reason for the change. Assuming that the unchanged component is compliant, if the reason for the change is to improve cost-effectiveness, increase testability, increase usability, or improve efficiency in some other way, the rework is type 2. Both type 0 or 1 and type 2 rework should increase the quality of the end product. However, type 0 or 1 also indicates inadequate quality in a current baseline. In practice, differentiating between type 0 or 1 and type 2 may be quite subjective. As discussed later, most of the metrics are insensitive to the categorization, but if the differentiation is consistently applied, it can provide useful insight. Collection and analysis of change metrics focus on type 0, 1, and 2 SCOs, which are collected and analyzed together.
Type 3 SCOs typically reflect a requirements change that redefines the customer expectations. Such changes have a broader impact and therefore require various levels of software and systems engineering as well as highly varying levels of regression testing. Because of this wide range of variability, type 3 SCOs are analyzed separately in these metrics. The data derived from type 0, 1, and 2 SCOs should provide a solid basis for estimating maintainability and the effort required for type 3 SCOs.
Whether SLOC provides a good metric for measuring software volume has always been controversial. (DeMarco calls this bang.) Jones identifies some of the precautions necessary when dealing with SLOC [Jones, 1994]. He goes so far as to say that "using lines of code for normalization of data involving multiple or different languages should be considered an example of professional malpractice." One point everyone agrees on is that whatever is used must be defined objectively and consistently to be of value for comparison. How the absolute unit of SLOC is defined is not as important as defining it consistently across all projects and all areas of a specific project. Requiring the use of a common counting tool forces standardization on a given definition.
The CCB is the governing body responsible for authorizing changes to a configured baseline product. At a minimum, members include the software development manager, the software assessment manager, and, for a contractual effort, a customer representative. The CCB decides on all proposed changes to configured products and approves all SCOs. The CCB is responsible for collecting the software quality metrics, analyzing trends objectively and subjectively, and proposing changes to the development process, tools, products, or personnel to improve future quality.
A configured baseline is a set of products that are subjected to change control through the CCB. Configured baselines may represent intermediate products that have completed design, development, and informal test, or final products that have completed formal test.
Was this article helpful?
What you need to know about… Project Management Made Easy! Project management consists of more than just a large building project and can encompass small projects as well. No matter what the size of your project, you need to have some sort of project management. How you manage your project has everything to do with its outcome.