General Discussion cont'd

DOE-HDBK-1204-97

After the test item format is determined, the number of test items to be developed should be

established. Some of the factors that significantly affect the number of test items to be

developed are the number and type of examinations on which the test items will be used, the

ease with which test items can be created, and how frequently a test item will be reused. The

initial number of test items selected for development should range from at least one to several

test items per learning objective.

Test items should be written after the format is selected and the number of test items is

determined. Test items should have answer keys written at this time.

Content validation is the process by which test items are determined to be consistent between

the content being learned and what is being tested. Each test item should be reviewed by

subject matter experts (SMEs). These validation reviews should be documented.

A test item bank should be developed and maintained current. The test items in the test bank

may be used to construct entry-level tests, progress tests, or after-instruction tests. A

tracking system should be developed to correlate test items to the corresponding learning

objective.

The following chapters offer guidance on how to select, construct, and score different

formats of written test items (e.g., short-answer, multiple-choice, matching, and essay).

Traditionally, test items that require the trainee to supply an answer (e.g., short-answer,

essay) have been considered "subjective"; test items requiring the trainee to select an

answer (e.g., multiple-choice, matching) have been considered "objective." If graders

require subject matter expertise to interpret the answers of test takers, the test item is

considered subjective. If the examination can be scored without having to interpret the

answer (e.g., machine scored), it is considered objective. An objective test item is defined

here as one in which: (a) there is only one correct answer, and (b) all qualified graders

would agree on the amount of credit allowed for any given trainee's answer.

Writing test items that are technically sound is a skillful and challenging task. Exam writers

must balance technical concepts with an array of test item writing principles to create a

single "good" test item. The test item should be one that reasonably assesses what a