The Use Of Lessons Learned

Risk issues that are analyzed to be medium or higher must be handled to the extent assets allow, to reduce their potential to adversely affect the program. All levels of management must be sensitive to hidden "traps" that may induce a false sense of security. If properly interpreted, these signals really indicate a developing problem in a known area of risk. Each trap is usually accompanied by several "warning signs" that show an approaching problem and the likelihood of failing to treat the problem at its inception.

The ability to turn traps into advantages suggests that much of the technical risk in a program can be actively handled via the risk handling control or transfer option, not merely watched and resolved after a problem occurs. In some instances it may pay to watch and wait. If the probability that a certain problem will arise is low or if the cost exceeds the benefits of "fixing" the problem before it happens, a do-nothing alternative (assumption risk handling option) may be advisable. Effective risk management makes selection of the do-nothing alternative a conscious decision rather than an oversight and may trigger an appropriate addition to the risk "watch list."

"Best practices" acknowledges that all of the traps have not been identified for each risk issue. The traps are intended to be suggestive, and other potential issues should be examined as they arise. It is also important to recognize that sources and types of risk evolve over time. Risks may take a long time to mature into problems. Attention must be properly focused to examine risks and lessons learned.

Lessons learned should be documented so that future project managers can learn from past mistakes.

Experience is an excellent teacher in risk management. Yet, no matter how hard we try, risks will occur and projects may suffer. As an example, we have over forty years of knowledge in going from new product development to production.7 We plan for risk man

7. Adapted from Transition from Development to Production. DoD 4245.7, Department of Defense, September 1985. These risk areas may occur on a variety of projects, but it may not be possible to take decisive action to deal with some of them until midway in the development phase.

agement, identify and analyze risk issues, and develop ways of handling and monitoring risks, but some types of risk issues commonly occur on projects that are mid to late in the development phase. Examples of these risk issues are:

Risk Issue: Design Process. The design process must reflect a sound design policy and proper engineering disciplines and practices—an integration of factors that influence the production, operation, and support of a system throughout its life cycle. Nevertheless, concepts are often selected, demonstrated, and validated with little thought given to the feasibility of producing a system employing those concepts. This omission is then carried forward into design, with voids appearing in manufacturing technology and absence of proven manufacturing methods and processes to produce the system within affordable cost. One of the most common sources of risk in the transition from development to production is failure to design for production. Some design engineers do not consider in their design the limitations in manufacturing personnel and processes. The predictable result is that an apparently successful design, assembled by engineers and highly skilled model shop technicians, goes to pieces in the factory environment when subjected to rate production. A design should not be produced if it cannot survive rate production without degradation.

Prevention. The potential to produce a system must be investigated carefully during the planning phase by means of appropriate producibility analyses. Voids in manufacturing technology projects and manufacturing methods and processes peculiar to the design of the specific system, subsystem, and components must be addressed during engineering development.

Risk Issue: Design Reviews. While most engineering development projects usually require formal design reviews, they often lack specific direction and discipline in the design review requirement, resulting in an unstructured review process that fails to fulfill either of the two main purposes of design review, which are (1) to bring additional knowledge to the design process to augment the basic program design and analytical activity and (2) to challenge the satisfactory accomplishment of specified design and analytical tasks needed for approval to proceed with the next step in the process.


• The customers and their contractors recognize that design reviews represent the "front line" where readiness for transition from development to production is decided ultimately. Design review policy, schedule, budget, agenda, participants, actions, and follow-up are decided in view of this foremost need.

• Design reviews should be included in all projects in accordance with existing customer requirements. A design review plan must be developed by the contractor and approved by the customer.

Risk Issue: Life. Life tests are intended to assess the adequacy of a particular equipment design when subjected to long-term exposure to certain operational environments. Due to the time-consuming nature of these tests, various methods have been used to accelerate test times by exposure to more stringent environments than those expected in actual operational use. These methods may give misleading results due to a lack of understanding of the acceleration factors involved.

Many projects are forced into conducting life tests after the systems are in use and before reliability requirements are achieved. As a result, life tests are performed after the start of production, and costly engineering change proposals (ECPs) and retrofit programs must be initiated in an attempt to "get well" with less than optimum design solutions.


• Include life testing in the overall system integrated test plan to ensure that testing is conducted in a cost-effective manner and to meet program schedules.

• Use test data from other phases of the test program to augment the system and subsystem life testing by reducing the time required to prove that reliability requirements are met.

• Use life test data from similar equipment, operating in the same environment, to augment the equipment life testing in order to gain confidence in the design.

Risk Issue: Manufacturing Plan. Involvement of production and manufacturing engineering only after the design process has been completed is a fundamental error and a major transition risk. Consequences of late involvement are: (1) an extended development effort required for redesign and retest of the end-item for compatibility with the processes and procedures necessary to produce the item and (2) lower and inefficient rates of production due to excessive changes in the product configuration introduced on the factory floor. Increased costs and schedule delays are the result of this approach.

Prevention. The following represent the key elements of a manufacturing plan:

• Master delivery schedule that identifies by each major subassembly the time spans, need dates, and who is responsible.

• Durable tooling requirements to meet increased production rates as the program progresses

• Special tools

• Special test equipment

• Assembly flowcharts

Risk Issue: Quality Manufacturing Process. The introduction of a recently developed item to the production line brings new processes and procedures to the factory floor. Changes in hardware or work flow through the manufacturing facility increase the possibility of work stoppage during rate production. Failure to qualify the manufacturing process before rate production with the same emphasis as design qualification—to confirm the adequacy of the production planning, tool design, manufacturing process and procedures—can result in increased unit costs, schedule slippage, and degraded production performance.


• The work breakdown structure, production statement of work, and transition and production plans do not contain any conflicting approaches. Any discrepancies among these documents are identified and resolved before production is started.

• A single-shift, eight-hour day, five-day workweek operation is planned for all production schedules during initial start-up. Subsequent manpower scheduling is adjusted to manufacturing capability and capacity consistent with rate production agreements.

• The drawing release system must be controlled and disciplined.

• The manufacturing flow must minimize tooling changes and machine adjustments and ensure that alternate flow plans have been developed.

• A mechanism must be established that ensures the delivery of critical items with long-lead time four to six weeks before required.

• All new equipment or processes that will be used to produce the item must be identified.

Risk Issue: Manpower and Personnel. Product development and support systems must be designed with as complete an understanding as possible of user manpower and personnel skill profiles. A mismatch yields reduced field reliability, increased equipment training, technical manual costs, and redesign as problems in these areas are discovered during demonstration tests and early fielding. Discovery of increased skill and training requirements late in the acquisition process creates a difficult catch-up problem and often leads to poor system performance.


• Manpower and skill requirements must be based on formal analysis of previous experience on comparable systems and maintenance concepts.

• Manpower cost factors used in design and support trade-off analyses must take into account costs to train or replace experienced personnel, as well as the true overhead costs.

Risk Issue: Training, Materials, and Equipment. On some programs, training requirements are not addressed adequately, resulting in great difficulty in operation and support of the hardware. Training programs, materials, and equipment such as simulators may be more complex and costly than the hardware they support. Delivery of effective training materials and equipment depends on the understanding of final production design configuration, maintenance concepts, and skill levels of personnel to be trained. On many programs, training materials and equipment delivery schedules are overly ambitious. The results include poor training, inaccuracies in technical content of materials, and costly redesign and modification of training equipment.


• Contractors must be provided with clear descriptions of user personnel qualifications and current training programs of comparable systems, to be used in prime hardware and training systems design and development.

• On-the-job training capability must be incorporated in the prime equipment design as a method to reduce the need for additional training equipment.

Project Management Made Easy

Project Management Made Easy

What you need to know about… Project Management Made Easy! Project management consists of more than just a large building project and can encompass small projects as well. No matter what the size of your project, you need to have some sort of project management. How you manage your project has everything to do with its outcome.

Get My Free Ebook

Post a comment