## Sensitivity to Criteria Weights

The evaluation framework shown in Table 9.2 reveals that the results are obviously sensitive to:

a. The selection of criteria b. The weighting of these criteria c. The ratings of each system

As an example of the criteria-weight sensitivity, let us assume that the criteria weights are changed as follows:

 Criteria From To 1 0.08 0.1 2 0.1 0.1 (no change) 3 0.13 0.1 4 0.09 0.1 5 0.12 0.1 6 0.07 0.1 7 0.11 0.2 8 0.1 0.2

With the same set of ratings, the total scores for each of the three alternatives now become

Thus, the modified criteria weights completely reverse the order of preference. This may have occurred, for example, if criteria 7 and 8 were related to cost and together comprise 40% of the total weight. If cost were considered that important, the preferred alternative would shift from C to A. This example leads to a simple but very important conclusion: No final system selection should be made without extensive sensitivity analyses!

### 9.8 MODELING AND SIMULATION

Most of the time, modeling and simulation techniques are employed in order to calculate technical performance measures (TPMs) and to carry out tradeoff studies between key measures and system parameters. The radar situation involving the detection of pulses in the presence of noise, discussed in Section 9.7.2, serves as a good example of how a model may be used to determine system performance.

The model of the detection process may be expanded to include the normal processing of pulses from the point at which they leave the transmitter to their reception at the threshold detector. Such a model may be constructed as a Parameter Dependency Diagram (PDD), a process developed by this author to model and analyze complex systems [9.9]. We start a PDD by identifying the key output parameters that we wish to compute. In the radar situation, these are the probability of detection, P(d), and the false-alarm probability,

P(fa), as cited earlier. For each of these outputs, we then ask the question: What do these parameters depend on? As shown in Figure 9.5, P(d) depends on three parameters: noise power (N), signal voltage (V), and detection threshold (T). In a similar vein, P(fa) depends on Nand T, but not V. All of these dependent parameters are also known as technical performance parameters (TPPs). If we continue to work backwards from these TPPs (N, V, and T), we can determine their dependent parameters until we come to the input signal, S(in), and the parameters on which it depends. This latter dependency is also known as the radar range equation, with the following TPPs:

• Gain of transmitting antenna, G(t)

• Target cross section strength, a

• Receiver processing power gain, G

• Receiver power processing losses, L

The blocks in the PDD implicitly represent relationships or equations that relate the input parameters to the output parameters. If we are operating in the frequency domain, the boxes can be thought of as the transfer functions relating inputs to outputs. In all cases, the PDD is constructed initially without knowing the precise relationship between inputs and outputs. The structure of the PDD, however, makes it very clear as to what the key parameters are and the known or unknown relationships among these parameters. If a relationship is currently unknown, then a modeling effort is required to determine

Figure 9.5. Illustrative parameter dependency diagram (PDD).

the necessary equations. The PDD is therefore a ''roadmap'' that explicitly shows the TPMs, the TPPs, and the known or unknown relationships between them. It is a performance ''model'' that represents how the key measures and parameters interrelate. Given the PDD and the necessary equations, the systems engineer is now in a position to carry out extensive trade-off studies and sensitivity analyses.

The preceding Parameter Dependency Diagramming (PDD) procedure is but one of many modeling techniques. If a model is particularly complex, the system engineer may wish to move to simulation, either by building a simulator to apply to the situation at hand or by using an existing, commercially available, simulation package. If workable in terms of the problem, the latter is highly recommended because there are numerous software packages available, at reasonable cost, to the systems engineering team. A list of such packages is provided in Exhibit 9.5 [9.14, 9.15]. For the reader who is interested in the perspectives of one of the industry leaders in modeling and simulation, it is recommended that the words of A. Pritsker be taken very seriously [9.17].

Exhibit 9.5: Selected Modeling and Simulation Software [9.14, 9.15]

Name of Software Package Builder/Publisher of Software

Achilles In-Motion Technology

ALSS II Productivity System

AutoMod AutoSimulations Inc.

BATCHES

Best-Network

Cinema Animation System

Batch Process Technologies Inc.

Best Consultants

SimSoft Inc.

Systems Modeling Corp.

CACI Products Company

Texas Tech University; Industrial En-

Extend FACTOR FMS+ +

gineering Department ISIM Simulation Imagine That Inc. Pritsker Corporation Texas Tech University; Industrial En-

GENETIK GPSS/H GPSS/PC HOCUS INMOD 1.8

GEMS-II

### General Simulation Sys.

gineering Department Gensym Corp. Lodestone II Inc. Prediction Systems Inc. Insight International Ltd. Wolverine Software Corp. Minuteman Software P-E International PLC Technical University—Sofia, Bulgaria

Name of Software Package

Builder/Publisher of Software

INSTRATA InterSIM ISEE-SIMNON ISI-PC

LANNET II.5

MAST Simulation Environ

Micro Saint & Animation micro-GPSS

MODSIM II

MOGUL

MOR/DS

NETWORK II.5

Packaging Lines Sim.Sys

Pascal Sim

PASION

PC Simula

PCModel

PERCNET

ProModel/PC

Proof Animation

QASE RT SES/workbench

SIGMA SIMAN

SIMFACTORY II.5

SIMNET II

SIMNON

SIMNON

SIMSCRIPT II.5

SIMSTARTER

SLAM/TESS

SLAMSYSTEM

Teamwork/SIM

XCELL+

Insight International Ltd. OLIM Holding Company Engineering Software Concepts Inc. Extech Ltd.

CACI Product Company

CMS Research Inc.

Micro Analysis/Design Simulation

Stockholm School of Economics CACI Product Company High Performance Software Inc. Holden-Day Publishing Company CACI Products Company Pritsker Corporation University of Southhampton, UK S. Raczynski, Mexico Simula a.s., Norway SimSoft

Mitchell & Gauthier Associates Production Modeling Corp.

International Wolverine Software Inc. AT&T Bell Laboratories Advanced System Technologies Inc. Scientific and Engineering Software. Inc.

The Scientific Press Inc. Systems Modeling Corp. CACI Products Company SimTec Inc.

SSPA Systems, Sweden Engineering Software Concepts Inc. CACI Products Company Network Dynamics Inc. Pritsker Corporation Pritsker Corporation Cadre Technologies Inc. Pritsker Corporation

Modeling and simulation is a specialized and rather complex subject, but it is essential that the systems engineering team master a variety of tools and techniques to be in a position to evaluate system performance on a quantitative basis. Until the system, or portions thereof, is actually built, there is really no other choice but to depend on modeling and simulation for performance assessments. The systems engineering team that faces this issue squarely will be competitive in the world of building large-scale systems. Those who do not are likely to be behind the power curve.

What you need to know about… Project Management Made Easy! Project management consists of more than just a large building project and can encompass small projects as well. No matter what the size of your project, you need to have some sort of project management. How you manage your project has everything to do with its outcome.

Get My Free Ebook