External Evaluations

DOE-HDBK-1201-97

Training materials should also be evaluated periodically. This should ensure that the

required changes are being incorporated. These changes may include changes to DOE

regulations, modifications to equipment or procedures, changes in job scope, and industry

operating experiences that impact the training.

External Evaluations

Levels III and IV focus on the impact of the training on the job (external). External

evaluation is the process that collects data from prior trainees, supervisors and managers,

and from many other outside sources that are beyond the actual domain of the training

program.

One method of collecting external evaluation data is to directly observe the graduate during

actual job performance. This method is the most direct approach to getting an answer to

the question of whether or not the graduate can perform the tasks on the job that he/she

was trained. Observations may be performed by a single evaluator or a team composed of

evaluators and subject matter experts. A checklist can be beneficial to aid in observations.

A method of collecting data from prior trainees and their supervisors is by using

questionnaires. Questionnaires are the least expensive approach for collecting data from

graduates and their supervisors. The validity of these questionnaires reflects how well the

evaluators prepared and distributed them. The questionnaire should include the tasks that

the trainees were trained to perform. Examples 4 and 5 represent this method.

Another method used to collect data from prior trainees and their supervisors is through

personal interviews. This method allows evaluators to collect more detailed information

than is possible on questionnaires. Evaluators can also follow-up on unclear responses and

can clarify any problems the graduate is having. Interviews should concentrate on

determining graduate proficiency. This is best accomplished by using a preplanned list of

questions.

Instructional quality data is collected during instruction, immediately after instruction, and at

various times after training to assure program maintenance. How data is collected is

influenced by the type of evaluation instrument used and by the training being evaluated.