Conduct First Run

DOE-HDBK-1078-94

Difficulty of the material

Length of the training

Amount of material covered

Clarity of the material

Terminology used in the material

Pace of the training

Structure of the material and sequence of training

Quantity and quality of practice exercises

Quality of the media

Relevance of the training to job performance.

For courses of lengthy duration it is not always feasible to conduct a

small-group evaluation. An alternative is to conduct small-group evaluations on

the most important segments of the course. Courses or segments of courses not

submitted to small-group evaluation should receive increased monitoring and

emphasis during the first run. See "Indicators of Potential Training

Program Weaknesses," "Post-training Questionnaire," and "Post-training

Interview" (Attachments 31 through 33, respectively) for examples of a

checklist, questionnaire, and interview form that can be used to collect data

during the small-group evaluation and the first run.

4.4.6 Conduct First Run. The first run verifies the usability of the training material

under intended conditions and confirms the revisions made to the material

during the technical review and small-group evaluation. Prior to conducting the

first run, material should receive formal approval by the training and referent

organization management. During the first run, learning and administrative

problems are noted and trainee comments on the training are obtained.

Techniques for data collection are the same as for small-group evaluation.

After collection, the data should be analyzed to improve applicable training

materials. If problems are identified during the tryout, retraining of the trainees

involved may have to occur once the materials are corrected.

4.4.7 Evaluate Data and Revise Material. Data collected during the small-group

evaluation and tryout are translated into findings, and decisions are made for

revising the training materials. Progress and post-test scores should be

collected and analyzed to determine if the intended learning outcome is reflected

in the learning objectives and their associated test items. If the training does not

produce the intended learning outcomes, revision of training materials should be

considered. It should be noted that faulty test items may not accurately

measure the intended learning outcome of the trainees. Those test items

consistently missed by trainees should be analyzed for faulty construction.

Supporting training materials should also be analyzed for clarity, completeness,

and technical accuracy.