Assessing the Impact of Uncertainty on Prioritization.

DOE-DP-STD-3023-98

express the variability by graphing accident frequency, likelihood, or probability as

a function of the number of fatalities, the number of cancer occurrences, or

another measure of accident consequences. Such variabilities are not typically

considered to be uncertainties; rather the term " ncertainty"refers to the fact that

either the risk model is not exact (modeling uncertainty) or that the parameters of

the input probability distributions are not exactly known (parameter uncertainty).

The RBP system and its underlying risk assessments should normally address

variabilities.

b. Types of Uncertainty. Two basic types of uncertainty within the RBP system

should be recognized:

1. Modeling uncertainty. The RBP system may produce inaccurate results if

important performance measures have been omitted from the aggregation

equation or if important dependencies among performance measures have not

been considered. Modeling uncertainty is difficult to assess quantitatively and

is frequently addressed by ensuring that the model is as complete as possible.

Bounding and/or subjective modeling techniques can be employed, when

warranted, to explore the quantitative implications of model limitations in the

more sophisticated applications of RBP. Documentation of a prioritization

project should explain the basis for selecting the decision analysis process

used in the project, including a rationale for including each performance

measure.

2. Parameter Uncertainty. The RBP system may produce inaccurate results if the

underlying data and information supplied to the aggregation equation is itself

uncertain. For example, the benefits or costs associated with a particular

decision option may not be exactly known; such information may be

communicated by providing (a) a range of values (that is, a minimum and a

maximum value) or (b) a probability distribution function.

c. Assessing the Impact of Uncertainty on Prioritization. It is important to understand

that the results of a prioritization (that is, an ordered list of decision options) may

be impacted by uncertainties and how they may be affected. It is desirable that

the treatment of uncertainties in the bottom line priority measures reflect all

sources of variability and uncertainty, to avoid misleading the user by reporting

only some contributors to the uncertainty of the bottom line. However,

distinguishing causal contributors to variability or uncertainty is often illuminating to

inform the user about the dominant factors limiting our ability to discriminate the

optimum decision alternative. When assessing uncertainty, it is recommended

that the guidance provided by the OMB be followed (References [k] and [l]).

Typically, the assessment of uncertainties has focused on understanding the

impacts of parameter uncertainty. The basic methods for assessing the impact of

parameter uncertainty include:

1. Sensitivity Analysis. A sensitivity analysis is performed by varying parameters

within the aggregation equation over a range of values and by observing the

effect on the prioritization results. Usually, the parameters are considered