Quality Indicators

The four quality indicators are based primarily on the measurement of software change across evolving baselines of engineering data (such as design models and source code). These metrics are developed more fully in Appendix C.

13.3.1 Change Traffic and Stability

Overall change traffic is one specific indicator of progress and quality. Change traffic is defined as the number of software change orders opened and closed over the life cycle (Figure 13-5). This metric can be collected by change type, by release, across all releases, by team, by components, by subsystem, and so forth. Coupled with the work and progress metrics, it provides insight into the stability of the software and its convergence toward stability (or divergence toward instability). Stability is defined as the relationship between opened versus closed SCOs. The change traffic relative to the release schedule provides insight into schedule predictability, which is the primary value of this metric and an indicator of how well the process is performing. The next three quality metrics focus more on the quality of the product.

Project Schedule

Figure 13-5. Stability expectation over a healthy project's life cycle

Project Schedule

Figure 13-5. Stability expectation over a healthy project's life cycle

13.3.2 Breakage and Modularity

Breakage is defined as the average extent of change, which is the amount of software baseline that needs rework (in SLOC, function points, components, subsystems, files, etc.). Modularity is the average breakage trend over time. For a healthy project, the trend expectation is decreasing or stable (Figure 13-6).

This indicator provides insight into the benign or malignant character of software change. In a mature iterative development process, earlier changes are expected to result in more scrap than later changes. Breakage trends that are increasing with time clearly indicate that product maintainability is suspect.

13.3.3 Rework and Adaptability

Rework is defined as the average cost of change, which is the effort to analyze, resolve, and retest all changes to software baselines. Adaptability is defined as the rework trend over time. For a healthy project, the trend expectation is decreasing or stable (Figure 13-7).

Released Baselines m a) o>

Implementation Changos

Project Schedule

Figure 13-6. Modularity expectation over a healthy project's life cycle

Released Baselines o

Implementation Changes

Project Schedule

Figure 13-7. Adaptability expectation over a healthy project's life cycle

Not all changes are created equal. Some changes can be made in a staff-hour, while others take staff-weeks. This metric provides insight into rework measurement. In a mature iterative development process, earlier changes (architectural changes, which affect multiple components and people) are expected to require more rework than later changes (implementation changes, which tend to be confined to a single component or person). Rework trends that are increasing with time clearly indicate that product maintainability is suspect.

13.3.4 MTBF and Maturity

MTBF is the average usage time between software faults. In rough terms, MTBF is computed by dividing the test hours by the number of type 0 and type 1 SCOs. Maturity is defined as the MTBF trend over time (Figure 13-8).

Early insight into maturity requires that an effective test infrastructure be established. Conventional testing approaches for monolithic software programs focused on achieving complete test coverage of every line of code, every branch, and so forth. In m

Released Baselines

Project Schedule

Figure 13-8. Maturity expectation over a healthy project's life cycle today's distributed and componentized software systems, such complete test coverage is achievable only for discrete components. Systems of components are more efficiently tested by using statistical techniques. Consequently, the maturity metrics measure statistics over usage time rather than product coverage.

Software errors can be categorized into two types: deterministic and nondeter-ministic. Physicists would characterize these as Bohr-bugs and Heisen-bugs, respectively. Bohr-bugs represent a class of errors that always result when the software is stimulated in a certain way. These errors are predominantly caused by coding errors, and changes are typically isolated to a single component. Heisen-bugs are software faults that are coincidental with a certain probabilistic occurrence of a given situation. These errors are almost always design errors (frequently requiring changes in multiple components) and typically are not repeatable even when the software is stimulated in the same apparent way. To provide adequate test coverage and resolve the statistically significant Heisen-bugs, extensive statistical testing under realistic and randomized usage scenarios is necessary.

Conventional software programs executing a single program on a single processor typically contained only Bohr-bugs. Modern, distributed systems with numerous interoperating components executing across a network of processors are vulnerable to Heisen-bugs, which are far more complicated to detect, analyze, and resolve. The best way to mature a software product is to establish an initial test infrastructure that allows execution of randomized usage scenarios early in the life cycle and continuously evolves the breadth and depth of usage scenarios to optimize coverage across the reliability-critical components.

As baselines are established, they should be continuously subjected to test scenarios. From this base of testing, reliability metrics can be extracted. Meaningful insight into product maturity can be gained by maximizing test time (through independent test environments, automated regression tests, randomized statistical testing, after-hours stress testing, etc.). This testing approach provides a powerful mechanism for encouraging automation in the test activities as early in the life cycle as practical. This technique could also be used for monitoring performance improvements and measuring reliability.

Project Management Made Easy

Project Management Made Easy

What you need to know about… Project Management Made Easy! Project management consists of more than just a large building project and can encompass small projects as well. No matter what the size of your project, you need to have some sort of project management. How you manage your project has everything to do with its outcome.

Get My Free Ebook


Responses

  • bailey
    What is measuring stability and modularity in SPM?
    3 years ago
  • Kye
    What are quality indicators in a project?
    2 years ago
  • Luwam
    What are software quality indices for indicating software quality?
    19 days ago

Post a comment