Strategic Planning for Training Evaluation

1. Strategic Planning for Training Evaluation Presented by: Nita C. Lescher, M.S.W., L.I.C.S.W. American Humane Association

2. When to Evaluate? Before, During, and After Training

3. Why Evaluate? Before Training • Identify needs or problems. • Identify specific job competencies to close gaps. • Identify the type of evaluation needed.

4. Why Evaluate? During Development of Training • Determine which staff should be the targets of training. • Provide feedback to curriculum designers on content, methods, and materials. • Determine if course delivery is consistent and follows curriculum specifications.

5. Why Evaluate? After Training • Note changes in participants’ attitudes and reactions to the course. • Assess trainees’ mastery of knowledge. • Assess trainees’ mastery of skills. • Assess transfer of new knowledge and skills to the job. • Measure whether or not performance gaps have begun to close. • Measure relationship of training to agency goals and client outcomes.

6. Strategic Plan for Training Evaluation Why do it? Specify clear relationships among agency goals, resources, purposes of training evaluation and decisions about what to evaluate.

7. Factors to Consider • Purpose of training • Cost of evaluation • Centrality of subject matter to competent job performance • Consequences of poorly developed/delivered training • Number of staff to be reached • Length of course • Legal mandates, public and political pressures • Agency goals

8. AHA Training Evaluation Model

9. Apple Peel Levels Level 1 = Course Evaluation of the training: • Content • Structure • Methods • Materials • Delivery

10. Apple Peel Levels Level 2 = Satisfaction (Happiness Sheets) Assesses trainees’ feelings about: • Trainer • Quality of material presented • Methods of presentation • Environment (e.g., room temperature)

11. Apple Peel Levels Level 3 = Opinion Assesses trainees’ attitudes/perceptions about: • Utilization of training • Own learning • Training’s value

12. Apple Peel Levels Level 4 = Knowledge Acquisition Assesses learning and recalling terms, definitions, and facts Level 5 = Knowledge Comprehension Assesses understanding concepts and relationships, recognizing examples in practice, and problem solving

13. Apple Peel Levels Level 6 = Skill Demonstration Assesses application of learned material to new and concrete situations within the training environment Level 7 = Skill Transfer Assesses application of new knowledge and skills outside the classroom and on the job

14. Apple Peel Levels Level 8 = Agency Impact Level 9 = Client Outcomes Level 10 = Community Impacts Assesses impact of training at agency, client, and community levels

15. Model for Successful Training Course Evaluation • Step 1: Identify the purposes of the evaluation. • Step 2: Determine the levels of evaluation needed. • Step 3: Develop an evaluation design. • Step 4: Select or develop instruments. • Step 5: Develop a data collection plan. • Step 6: Pilot and revise instruments and procedures. • Step 7: Collect evaluation data. • Step 8: Analyze evaluation data. • Step 9: Report results. • Step 10: Provide feedback/disseminate results.