Guideline 7.3-- Defining Acceptance Criteria.

DOE-DP-STD-3023-98

9.7.3

Guideline 7.3-- Defining Acceptance Criteria. The quality assurance (QA) and

validation criteria applied to the RBP system should be documented.

Discussion. The QA and validation criteria applied to an RBP effort should be

commensurate with the scale of the effort. The general goal is to ensure that the RBP

system has been properly applied and yields results that meet the original purposes

defined by the user. The QA and validation checks should verify that (1) assumptions

have been documented, (2) the significance of assumptions and values of parameters

used in the RBP system have been adequately examined and described, (3) there are

no significant signs of bias, and (4) any interdependencies among decision options

have been defined and properly handled.

9.7.4

Guideline 7.4-- Assuring Quality. Documentation of the prioritization model, input data

(work packages), and the preliminary RBP prioritization results should be subject to

QA and other validation checks before further use of the preliminary results.

Deficiencies should be corrected prior to proceeding with the remainder of the RBP

process.

Discussion. The QA and validation checks provide confidence that the prioritization

model was properly implemented and that the model produced results that pass tests

for reasonableness before the results are compiled for presentation to the decision

makers who requested the RBP. If scores were generated by a " coring team,"some

subset of that team can be used as the quality assurance team provided that

individuals do not review their own work. To counter and minimize " aming,"peer

review or other such techniques should be pursued. Peer review can ensure that the

highest professional standards are maintained. Therefore, procedures should be

developed to maximize the use of quality assurance and validation checks.

9.7.5

Guideline 7.5-- Documentation. The report produced at the conclusion of an RBP

effort should thoroughly define the original user requirements (purpose statement and

decision options), the RBP process applied, the input data used, and the results

derived. The results should be presented in a form that meets the original

requirements specified by the user.

Discussion. There are two key aspects to documentation. One is to provide the

necessary documentation to allow the decision maker to have confidence in using the

results as a basis for decisions. The second is to defend the results to a broad

audience of peer reviewers and interested parties.

a. Documentation to Support Decision Makers. An RBP system is useful only to the

extent that its outputs are relevant in the context of an actual decision-making

effort. A well-designed RBP process allows a wide range of potentially useful

outputs to be generated for decision makers. Specific user needs are defined at

the start of the RBP process and are captured in the decision options defined by

the end user. The RBP results should be selected and presented to maximally

achieve the purposes for which the system was developed. Thus, the purpose