In an attempt to overcome some of the disadvantages of profitability models, particularly their focus on a single decision criterion, a number of evaluation/selection models that use multiple criteria to evaluate a project have been developed. Such models vary widely in their complexity and information requirements. The examples discussed illustrate some of the different types.
Unweighted 0-1 Factor Model A set of relevant factors is selected by management. These are usually listed in a preprinted form, and one or more raters score the project on each factor depending on whether or not it qualifies for that individual criterion. The raters are chosen by senior managers, for the most part from the rolls of senior management. The criteria for choice are a clear understanding of organizational goals and a good knowledge of the firm's potential project portfolio. Figure 2-2 shows an example of the rating sheet for an unweighted, 0-1 factor model.
The columns of Figure 2-2 are summed and those projects with a sufficient number of qualifying factors may be selected. The main advantage of such a model is that it uses several criteria in the decision process. The major disadvantages are that it assumes all criteria are of equal importance and it allows for no gradation of the degree to which a specific project meets the various criteria.
Unweighted Factor Scoring Model The second disadvantage of the 0-1 factor model can be dealt with by constructing a simple linear measure of the degree to which the project being evaluated meets each of the criteria contained in the list.
No increase in energy requirements x
Potential market size, dollars x
Potential market share, percent x
No new facility required x
No new technical expertise required x
No decrease in quality of final product X
Ability to manage project with current personnel x
No requirement for reorganization x
Impact on work force safety x
Impact on environmental standards x
Rate of return more than 15% after tax x
Estimated annual profits more than $250,000 x
Time to break-even less than 3 years x
Need for external consultants x
Consistency with current lines of business x
Impact on company image With customers x
With our industry x
Figure 2-2: Sample project evaluation form.
The x marks in Figure 2-2 would be replaced by numbers. Often a five-point scale is used, where 5 is very good, 4 is good, 3 is fair, 2 is poor, I is very poor. (Three-, seven-, and 10-point scales are also common.) The second column of Figure 2-2 would not be needed. The column of scores is summed, and those projects with a total score exceeding some critical value are selected. A variant of this selection process might select the highest-scoring projects (still assuming they are all above some critical score) until the estimated costs of the set of projects equaled the resource limit. The criticism that the criteria are all assumed to be of equal importance still holds.
The use of a discrete numeric scale to represent the degree to which a criterion is satisfied is widely accepted. To construct such measures for project evaluation, we proceed in the following manner. Select a criterion, say, "estimated annual profits in dollars." For this criterion, determine five ranges of performance so that a typical project, chosen at random, would have a roughly equal chance of being in any one of the five performance ranges. (Another way of describing this condition is: Take a large number of projects that were selected for support in the past, regardless of whether they were actually successful or not, and create five levels of predicted performance so that about one-fifth of the projects fall into each level.) This procedure will usually create unequal ranges, which may offend our sense of symmetry but need not concern us otherwise. It ensures that each criterion performance measure utilizes the full scale of possible values, a desirable characteristic for performance measures.
Consider the following two simple examples. Using the criterion just mentioned, "estimated annual profits in dollars," we might construct the following scale:
Score Performance Level
5 Above $1,100,000
1 Less than $200,000
As suggested, these ranges might have been chosen so that about 20 percent of the projects considered for funding would fall into each of the five ranges.
The criterion "no decrease in quality of the final product" would have to be restated to be scored on a five-point scale, perhaps as follows:
Score Performance Level
The quality of the final product is:
5 significantly and visibly improved
4 significantly improved, but not visible to buyer
3 not significantly changed
• 2 significantly lowered, but not visible to buyer
This scale is an example of scoring cells that represent opinion rather than objective (even if "estimated") fact, as was the case in the profit scale.
Weighted Factor Scoring Model When numeric weights reflecting the relative importance of each individual factor are added, we have a weighted factor scoring model. In general, it takes the form
Si = the total score of the ith project,
Si,- = the score of the ith project on the j'th criterion, and
The weights, wmay be generated by any technique that is acceptable to the organization's policy makers. There are several techniques available to generate such numbers, but the most effective and most widely used is the Delphi technique. The Delphi technique was developed by Brown and Dalkey of the Rand Corporation during the 1950s and 1960s |15|. It is a technique for developing numeric values that are equivalent to subjective, verbal measures of relative value. (The method is also useful for developing technological forecasts. For a description of the technique see Appendix B and also reference |31j in the bibliography to that appendix.) The method of successive comparisons (or pairwise comparisons) may also be used for the same purpose. Originally described by Churchman, Ackoff, and Arnoff in their classic text on operations research (10], this technique asks the decision maker to make a series of choices between several different sets of alternatives. A set of numbers is then found that is consistent with the choices. These numbers can serve as weights in the scoring model. For an example of the use of this method, see [18). Another popular and quite similar approach is the Analytic Hierarchy Process, developed by Saaty, see 154, 65] for details.
When numeric weights have been generated, it is helpful (but not necessary) to scale the weights so that
Was this article helpful?
What you need to know about… Project Management Made Easy! Project management consists of more than just a large building project and can encompass small projects as well. No matter what the size of your project, you need to have some sort of project management. How you manage your project has everything to do with its outcome.