Characteristics For Evaluating, Implementing, and/or Comparing Risk Based

DOE-DP-STD-3023-98

is the measure of value. The more preferred the level of performance, the higher the value " ."

Furthermore, the differences in the values of " "produced under various levels of performance

" "indicate by how much the higher levels of performance are preferred.

Scoring. The process of determining the input parameter values required by the RBP model to

yield the performance result for each activity. These parameters are used in the aggregation

equation inputs, specifically those associated with measuring risk (i.e., probability and

consequence).

Stakeholders. Interested or affected parties. (Reference [f]).

8. CHARACTERISTICS FOR EVALUATING, IMPLEMENTING, AND/OR COMPARING RISK-

BASED PRIORITIZATION SYSTEMS

Eight characteristics shall be used for the purpose of evaluating the quality of a prioritization

system: (1) logical soundness, (2) completeness, (3) accuracy, (4) acceptability, (5) practicality,

(6) effectiveness, (7) defensibility, and (8) quantification of costs and benefits. These

characteristics are described below.

The degree to which a prioritization system must achieve each of the eight characteristics

depends on the application. For example, some situations demand highly accurate results, while

rough approximations are acceptable in others. Ensuring the adequacy of an RBP system

requires:

a. Determining necessary and desired levels of system performance with regard to each of

the characteristics.

b. Evaluating the capabilities of candidate RBP system designs with respect to those

characteristics.

c. Choosing a system design that achieves necessary levels of performance and desired

levels of performance with regard to each characteristic.

Guidelines are provided in section 9 to assist with meeting and properly balancing these criteria.

Some of the guidelines support more than one characteristic, so it may be useful to think of the

guidelines as a single set.

8.1 Logical Soundness. The system should be able to produce results and insights via a

defensible decision rule that adheres to the basic principles of logic. Such a rule should be

justifiable in terms of theoretical arguments or empirical evidence, sensitive to the various

aspects of the problem, unbiased, and reliable in the sense that independent applications to

the same problem would produce the same results.

8.2 Completeness. The RBP system should strive to account for all relevant and important

decision objectives that discriminate among potential alternatives. The system should allow

for inclusion of additional decision objectives that may arise during implementation of the

process. Objectives important to interested parties in the decision process (e.g.,

stakeholders, participants) should be addressed by the system. Users of the RBP system

should ensure that these objectives are included in the implementation of the system and the

documentation of the results.

8.3 Accuracy. The RBP system should minimize the potential for cognitive and motivational

biases that could inappropriately skew the results in favor of particular decision options. It

should be possible to demonstrate that any reasonable team with the same input data would

produce a similar result.