Variation and Uncertainty

Everyone knows that project tasks have a certain amount of inherent variation and frequently uncertainty about the amount of that variation. The very definition of a project says you have not done this task before; it is unique. At least, you have not performed all of the tasks the same way you will in this particular project. In order to complete the project successfully, you must account for this variation and uncertainty. People's ability to estimate accurately depends on a number of factors. There is substantial evidence to indicate that people tend toward overconfidence in their belief in the accuracy of their estimates [14]. It is unlikely that most project tasks can be estimated more accurately than Â±20%.

We have people estimate a very simple task as part of our training classes. Nearly all of the participants in the exercise agree that the task is much simpler than most of their project tasks. They also agree that the ability of the other people in the room to estimate this task should be as good as, or better than, the ability of their project estimators to estimate project estimates. The range of the estimates usually is several hundred percent of the mean. The standard deviation is usually on the order of 30% of the mean. Figure 1.4 illustrates combined results from several of these exercises.

Figure 1.5 illustrates the expected general behavior of the accuracy of a single task estimate as a function of the amount of effort put into creating the estimate. The accuracy scale presents the accuracy as a percentage of the mean estimate, so a perfectly accurate estimate has an uncertainty of zero. An estimate with no effort at

Activity duration estimates

60 80 Minutes

Figure 1.4 Estimate uncertainty for a very simple project task illustrates the typical range of real uncertainty.

Estimate uncertainty

Estimate uncertainty

Estimate cost as a fraction of project cost

Figure 1.5 Estimated accuracy generally increases with the effort applied to the estimate, up to a limit determined by the process involving the subject of the estimate.

Estimate cost as a fraction of project cost

Figure 1.5 Estimated accuracy generally increases with the effort applied to the estimate, up to a limit determined by the process involving the subject of the estimate.

all should have an accuracy of at least 100% on the down side and could be orders of magnitude (hundreds of percentage points) too low. The curve illustrates that the accuracy should generally improve as more effort is put toward the estimate. A lower limit usually limits improvement due to the inherent variation in the process that will produce the task result. This lower limit, described further in Chapter 2, is called common-cause variation. No matter how much more effort you put into the estimate, you can never do better than the common-cause variation of the process that produces the result of the task. You can only reduce common-cause variation by changing the task process.

Consider two regions of Figure 1.5, divided by the vertical dotted line. To the right of the line, adding more effort to the estimate does not significantly improve the accuracy of the estimate. If you are far to the right of the line, reducing the effort should not have much impact on uncertainty. Estimates to the left of the line show increasing sensitivity to the amount of effort applied. Small reductions in the applied effort will greatly increase the uncertainty of the estimate. Small increases in the effort will significantly improve the estimate.

Assuming that the tasks you have identified in your project plan identify deliverables (clean handoffs from one primary resource type to the next), the effect on overall plan uncertainty that will obtain from subdividing tasks to your project plan depends on the region in which you are operating. With a fixed level of investment in the estimate, if you are well to the right of the line, adding more tasks (which reduces the effort per task) may increase the accuracy of the overall plan. The reason is that the accuracy of the overall plan improves as the plan is divided into more equal-sized pieces, if the accuracy of the individual pieces is the same.

If the amount of estimating effort you can afford puts you near, or to the left of, the vertical line in Figure 1.5, adding more tasks to your plan can decrease the accuracy of the overall plan. The reason is that the increasing uncertainty in each task estimate can be much greater than the statistical benefit of more individual tasks.

From the alleged causes of project failure posed so far, you might deduce that uncertainty causes projects to fail. If this were the case, you should predict that all projects with uncertainty will fail. Based on the definition of a project and our understanding of the real world, all projects have uncertainty. Therefore, you might predict that all projects would fail. Many do, but not all. Furthermore, there is evidence that some projects succeed despite extreme uncertainty. Goldratt describes one airplane project that defies this prediction in Critical Chain. The designers developed an airplane with unprecedented capabilities in eight months instead of the ten years such developments normally take. There are other cases. The United States did succeed to meet President Kennedy's objective to put a man on the moon by the end of the decade. The moon project was one of the most uncertain projects ever undertaken by man. Creation of the atomic bomb was a similarly uncertain project completed in a remarkably short time.

There have been substantial efforts to reduce the uncertainty in project estimates and the variation in performing project tasks. There are excellent tools for estimating projects and project tasks that no doubt help improve the accuracy of estimates and, more importantly, collect data to estimate the variation in project tasks. There have been improvements in performing project tasks, using approaches such as Six Sigma. Unfortunately, the project-failure data includes companies that have applied such techniques to minimize variation. Collectively, they have not made much difference.

A cornerstone of the scientific method is that scientists can never prove that any scientific theory or law will continue to work in the future, but they can disprove a theory with just one proper test. More than one instance proves that uncertainty itself cannot be the cause of project failure.

If simple uncertainty does not meet the test of explaining project failure, can you modify the theory to fit the known evidence? You know that some projects use different ways to manage uncertainty. For example, the Apollo project managed risk by hiring three companies to produce three different solutions for high-risk developments. They chose one as the primary path, but had two backups in case the primary path failed. They planned on much test and retest. (And they had plenty of spectacular failures along the way.) While this is an expensive way to manage uncertainty, it worked. Goldratt used thinking like the above to pose the hypothesis that it is failure to effectively manage uncertainty that causes most project failures. Chapter 3 examines this hypothesis in depth. If he is right, the direction of the solution is to create a different project system more able to manage uncertainty.