Open-Reference Test Items cont'd

DOE-HDBK-1204-97

Test items should contain unique or varied circumstances that the trainees

have not previously encountered. This makes the test item a true indicator of

the trainee's ability to apply knowledge through the use of the references versus

merely remembering an application from an earlier training session. Familiarity

with routine applications of a reference can lower the learning level to simple

recall.

If a test item has references provided, it is not appropriate to make the test item

more or less difficult than a comparable closed book test item. While very

difficult test items may be useful in differentiating among the most able trainees,

they are not appropriate for job or task qualification. Do not make an open

reference test item easier or more difficult than other formats.

Keep test item requirements as close to real life situations as possible. It is

important the test items address the use of references in a context similar to

that found in the job environment. While providing a chart from a handbook is

good, giving the trainees handbooks and requiring that the chart be found can

be even better. Keep the content of open reference test items valid.

Care must be taken in the development of open-reference test items. Under

certain circumstances, these may become double jeopardy test items. In other

words, if the test developer is testing the ability to use a graph vice locate it,

the trainees will not be able to demonstrate use of the graph if they cannot find

it.

An open-reference test item is the most challenging kind of test item to develop because it

should challenge the highest cognitive level of the trainee. These test items are the result

of a great deal of effort on the part of the developer. Several revisions and reviews by

others are usually required to refine the test item into a quality test item that will challenge

the trainee's decision making ability.