Open-Reference Test Items

DOE-HDBK-1204-97

13. OPEN-REFERENCE TEST ITEMS

Training environments use several teaching aids that promote trainee learning. Many of

these aids (e.g., textbooks, manuals, and student guides) assist the trainee in mastering

the required subject matter during the initial training phase. The trainee is expected to

comprehend the material and commit important facts to memory. However, there are many

references (e.g., tables, charts, schematics, and procedures) trainees do not need to

commit to memory (and should not) but have to interpret while performing their job.

To test trainee competence in the use of this material, test items that require the use of

references to solve a problem or to reach a conclusion should be developed. This requires

that the reference, or a sufficient subset of the reference, be provided to the trainee during

administration of the test. This use of reference materials during testing is referred to as

open-reference or open book testing.

The test developer should determine which references and their applications are necessary

after reviewing the learning objectives and the test specifications. While the open

reference test is essentially no different than other written tests, there are several points to

consider when using this method.

References are considered tools trainees use to solve a problem or reach a

conclusion. The test items should not directly test knowledge of the references

(this should be done in a closed book section); rather, it should test for proper

use and application of the references.

Open-reference test items should not be a "direct lookup" in a reference. For

example, asking the specific setpoint for a protective action or the horsepower

of a motor are direct lookups. Rather the reference material should be an aid

in going through a series of knowledge-based decisions in order to arrive at a

logical conclusion or answer.

Open-reference test items should test the trainee's ability to locate, use, and

apply the information found in the references.