Developing Data and Information About Uncertainties.

DOE-DP-STD-3023-98

individually as it is difficult to interpret the results if combinations of parameters

are simultaneously varied. Sensitivity analysis is typically used as an initial

step in understanding uncertainties to indicate how the parameters within the

aggregation equation affect the prioritization results.

2. Uncertainty Analysis. An uncertainty analysis is performed by developing

probability distribution functions for the parameters within the aggregation

equation, and subsequently propagating these distributions through the

aggregation equation. The most difficult part of performing any uncertainty

analysis is the formulation of the underlying parameter distributions. A variety

of techniques have been developed to propagate the parameter distributions

through the aggregation equation (such as Monte Carlo analysis, meta-

analysis, discrete probability distributions, and the method of moments).

d. Developing Data and Information About Uncertainties. Historical data should be

used to gain insight about the extent of parameter uncertainties; such information

can be analyzed, interpreted, and communicated using standard statistical

methods. If no relevant historical data exist, expert opinion may be solicited and

processed using a variety of methods. There are a variety of formal methods for

eliciting and combining expert judgments of data distribution characteristics, such

as Delhi methods, that should be considered for use as appropriate. Limited

historical data may be combined with expert opinion using Bayesian statistical

methods. The information should be documented for peer review and for future

modeling use.

9.2.6

Guideline 2.6-- Development of Performance Measurement Scales. The RBP system

should be developed in such a way that each performance measure can reflect either

a benefit increase (desirable) or a benefit decrease (undesirable).

Discussion. The implementation of a given decision option can create either an

increase or a decrease in the benefit and/or cost associated with each performance

measure; accordingly, each performance measure scale should be capable of

reflecting either effect. Within a prioritization system, the only way to accurately

assess the net benefit of a decision option is to properly account for both the aspects

of an activity that move one closer to meeting a decision objective and those that

move one farther away. For example, consider an RBP system containing a

performance measure related to public health and safety that is used to assess two

decision options. Implementation of the first decision option may greatly reduce the

risk to worker health and safety (a large increase in benefit associated with this

objective), while it also may somewhat increase the risk to public health and safety (a

small decrease in benefit associated with this objective). The RBP system must be

capable of reflecting these effects, which is achieved through proper construction of

the performance measure scales.

Therefore, the performance measurement scales should reflect both direction

(increase or decrease in benefits) and magnitude (how much increase or decrease in

benefit). A simple way to achieve this need is to assign positive performance