Piloting the Performance Test

DOE-HDBK-1205-97

When the performance test is constructed, scoring procedures are developed. A detailed,

step-by-step description of required performance provides an effective scoring procedure for

some tasks. Action steps or elements required in the performance test usually are prepared

in checklist form, and the trainee is required to follow each step without deviation. For other

tasks, the product of performance (i.e., a tangible result) should be measured. In

developing a scoring procedure for this type of performance test, scorable characteristics

must be defined to distinguish clearly between satisfactory and unsatisfactory performance.

Scoring methods must adhere to administrative and instructional guidelines and reflect the

evaluation standards. If an evaluation standard is "without error" a yes/no checklist should

be used. If the standard implies some range of acceptable performance, a rating scale may

be used. However, rating scales introduce greater subjectivity and are more difficult to use,

to interpret, and to back up than a yes/no checklist. If sequence is important, identify this on

the performance test and provide proper scoring guidance.

Establish cutoff scores to meet the performance standards. Percentages are the least

preferred method. Failure of a performance test should be determined by the failure of any

critical step or the failure to follow required sequences. The cutoff score for any behavior

should be based on a level of performance that is achievable by the average person.

All methods of scoring should be consistent with policy, procedures, specifications, and

needs. The method should also adhere to instructional guidelines, such as testing the

objectives, and should clearly distinguish between satisfactory and unsatisfactory

performance.

Piloting the Performance Test

The purpose of piloting a performance test is to ensure that the "bugs" have been worked

out of the test. The pilot should be conducted under the conditions required for actual job

performance or the same conditions under which the trainee will be tested.