Following an elicitation session the analyst needs to provide feedback to the assessor about the combined judgements, if this was not possible during the elicitation session. This may lead to the assessor making changes to judgements made in the elicitation session.
Where elicitation of a variable involves more than one assessor, it is necessary to aggregate these judgements. This may involve group meetings to explore the basis for consensus judgements or resolve disagreements. Keeney and van Win-terfeldt (1991) found that whether or not substantial disagreements existed among expert assessors, there was almost always agreement among them that averaging of probability distributions (which preserved the range of uncertainties) was an appropriate procedure to provide information for a base case analysis.
It should be clear from the foregoing discussion that probability encoding is a non-trivial process that needs to be taken seriously for credible results. To be effective the encoding process needs to be carefully planned and structured and adequate time devoted to it. The complete process should be documented as well as the elicitation results and associated reasoning. For subsequent use, documentation should be presented in a hierarchical level of detail to facilitate reports and justification of results in appropriate levels of detail for different potential users. In all of these respects the encoding process is no different from other aspects of the risk analysis and management process.
Was this article helpful?