Peer inspections are frequently overhyped as the key aspect of a quality system. In my experience, peer reviews are valuable as secondary mechanisms, but they are rarely significant contributors to quality compared with the following primary quality mechanisms and indicators, which should be emphasized in the management process:
• Transitioning engineering information from one artifact set to another, thereby assessing the consistency, feasibility, understandability, and technology constraints inherent in the engineering artifacts
• Major milestone demonstrations that force the artifacts to be assessed against tangible criteria in the context of relevant use cases
• Environment tools (compilers, debuggers, analyzers, automated test suites) that ensure representation rigor, consistency, completeness, and change control
• Life-cycle testing for detailed insight into critical trade-offs, acceptance criteria, and requirements compliance
• Change management metrics for objective insight into multiple-perspective change trends and convergence or divergence from quality and progress goals
Although I believe that inspections are overemphasized, in certain cases they provide a significant return. One value of inspections is in the professional development of a team. It is generally useful to have the products of junior team members reviewed by senior mentors. Putting the products of amateurs into the hands of experts and vice versa is a good mechanism for accelerating the acquisition of knowledge and skill in new personnel. Gross blunders can be caught and feedback can be appropriately channeled, so that bad practices are not perpetuated. This is one of the best ways for junior software engineers to learn.
Inspections are also a good vehicle for holding authors accountable for quality products. All authors of software and documentation should have their products scrutinized as a natural by-product of the process. Therefore, the coverage of inspections should be across all authors rather than across all components. Junior authors need to have a random component inspected periodically, and they can learn by inspecting the products of senior authors. Varying levels of informal inspection are performed continuously when developers are reading or integrating software with another author's software, and during testing by independent test teams. However, this "inspection" is much more tangibly focused on integrated and executable aspects of the overall system.
Finally, a critical component deserves to be inspected by several people, preferably those who have a stake in its quality, performance, or feature set. An inspection focused on resolving an existing issue can be an effective way to determine cause or arrive at a resolution once the cause is understood.
Notwithstanding these benefits of inspections, many organizations overemphasize meetings and formal inspections, and require coverage across all engineering products. This approach can be extremely counterproductive. Only 20% of the technical artifacts (such as use cases, design models, source code, and test cases) deserve such detailed scrutiny when compared with other, more useful quality assurance activities. A process whose primary quality assurance emphasis is on inspections will not be cost-effective. Several published studies emphasize the importance and high ROI of inspections. I suspect that many of these studies have been written by career quality assurance professionals who exaggerate the need for their discipline. I am frequently a lone voice on this topic, but here is my rationale.
Significant or substantial design errors or architecture issues are rarely obvious from a superficial review unless the inspection is narrowly focused on a particular issue. And most inspections are superficial. Today's systems are highly complex, with innumerable components, concurrent execution, distributed resources, and other equally demanding dimensions of complexity. It would take human intellects similar to those of world-class chess players to comprehend the dynamic interactions within some simple software systems under some simple use cases. Consequently, random human inspections tend to degenerate into comments on style and first-order semantic issues. They rarely result in the discovery of real performance bottlenecks, serious control issues (such as deadlocks, races, or resource contention), or architectural weaknesses (such as flaws in scalability, reliability, or interoperability). In all but trivial cases, architectural issues are exposed only through more rigorous engineering activities such as the following:
• Analysis, prototyping, or experimentation
• Constructing design models
• Committing the current state of the design model to an executable implementation
• Demonstrating the current implementation strengths and weaknesses in the context of critical subsets of the use cases and scenarios
• Incorporating lessons learned back into the models, use cases, implementations, and plans
Achieving architectural quality is inherent in an iterative process that evolves the artifact sets together in balance. The checkpoints along the way are numerous, including human review and inspections focused on critical issues. But these inspections are not the primary checkpoints. Early life-cycle artifacts are certainly more dependent on subjective human review than later ones are. Focusing a large percentage of a project's resources on human inspections is bad practice and only perpetuates the existence of low-value-added box checkers who have little impact on project success. Look at any successful software effort and ask the key designers, testers, or developers about the discriminators of their success. It is unlikely that any of them will cite meetings, inspections, or documents.
Quality assurance is everyone's responsibility and should be integral to almost all process activities instead of a separate discipline performed by quality assurance specialists. Evaluating and assessing the quality of the evolving engineering baselines should be the job of an engineering team that is independent of the architecture and development team. Their life-cycle assessment of the evolving artifacts would typically include change management, trend analysis, and testing, as well as inspection.
Was this article helpful?
What you need to know about… Project Management Made Easy! Project management consists of more than just a large building project and can encompass small projects as well. No matter what the size of your project, you need to have some sort of project management. How you manage your project has everything to do with its outcome.