Evaluation of Training

1. Evaluation of Training NASC Tara Prasad Kharel

2. Contents • Introduction • Importance • Process • Models

5. What should we ask? “Why Evaluate Training?” or “Can we afford not to evaluate training?”

6. What is evaluation? • Process of establishing a worth of something • The worth means the value, merit or excellence • It is state of mind, rather than a set of techniques

7. What is training evaluation? • Measuring the effectiveness of training. • The extent to which the objectives of training have been achieved • A systematic process to analyze if training programs and initiatives are effective and efficient

9. Why evaluation? • To justify the need of training programme • To identify the strengths and weaknesses of training programme • To provide feedback on the effectiveness of the training activities • To improve the future training programme • To determine the competence level of the participants • To make compliance with the overall objectives of the organization

10. Benefits of Evaluation • Improved quality of training activities • Improved ability of the trainers to relate inputs to outputs • Better discrimination of training activities • Better co-operation between trainers and managers in the development of staff • Evidence of the contribution that training and development are making to the organization

11. Equip with artilleries Keep three questions in mind • What to evaluate? - Content • When to evaluate? - Time • How to evaluate? - Process/method

12. Five Steps of Training Evaluation • Step 1: Identify the Purposes of Evaluation • Step 2: Select Evaluation Method • Step 3: Design Evaluation Tools • Step 4: Collect Data • Step 5: Analyze and Report Results

13. Identify Purposes of Evaluation • Why do we want to evaluate training programs? • To learn from Experience for Future Improvement • To increase Accountability • To identify the strengths and weaknesses. • To compare the costs and benefits. • To test the clarity and validity of tests, cases and exercises. • To reinforce the key points made to the participants. • To determine whether the program was an appropriate solution for the specific need. • To establish a database that can assist in making decisions.

14. Select Evaluation Method (Models) • Donald Kirkpatrick’s model • Reaction • Learning • Behavior • Results • CIRO Model

15. Design Evaluation/Data Collection Tools • Questionnaires • Surveys • Tests • Interviews • Focus group discussions • Observations • Performance records

16. Analyze and Report Results • Data Input • Data Analysis • Frequency distribution • Using figures • Tables • Charts • Reporting • Who needs to know what? • Forms of communicating • Evaluation report

17. Evaluation report outline • Summary • Training Program Description • Training Evaluation Design and Methods • Findings and Results • Recommendations • Appendices

18. Summary • Purpose of training evaluation • Training evaluation audiences • Major findings and recommendations • Training Program Description • Training Program background • Training Program goals/objectives • Training Program participants • Training Program activities • Training Evaluation Design and Methods • Purpose of the training evaluation • Training evaluation designs • Data collection methods

19. Findings and Results • Description of how the findings are organized (e.g., by evaluation questions, themes/issues) • Results of analyses of quantitative and/or qualitative data collected • Recommendations • Recommendations for action based on these conclusions • Appendices • List of participants • List of Training materials • Questionnaires, pre/post tests • Program expenditure summary

21. Levels of Evaluation

23. Donald Kirkpatrick’s model • Reaction level evaluation – Immediate reaction - How was the training overall? (Effectiveness) • Learning level evaluation - What knowledge and abilities did participants learn at the training? • Performance/ behaviour level evaluation - How have participants applied the skills they learned? • Impact level evaluation - What was the effect on the organization?

24. Reaction Level Evaluation • What to evaluate? • Overall impression of the training programme(objectives, contents, resource persons, methodologies, training materials and aids, logistic support, food, accommodation etc.) • When to evaluate? • Generally, at the end of training programme • Mid-way of the training programme • How to evaluate? • Questionnaire, general discussion, one-to-one q/a

26. Learning Level Evaluation • What to evaluate? • The extent of which the learning was transferred – Knowledge, skills etc. • How to evaluate? • Pre/post test • Learning test/written test • Performance test • When to evaluate? • At the end of training programme • Mid-way of training programme

28. Performance Level Evaluation • What to evaluate? • Changes of job behaviour/performance because of training • When to evaluate? • After 6 months • How to evaluate? • Observation • Interview with the participant/immediate boss • Using checklist of work procedure • Interview with the stakeholders • Performance evaluation

29. Impact Level Evaluation • What to evaluate? • Measuring the effects of performance of trainees on efficiency of organisation • When to evaluate? • After 1 – 2 years • How to evaluate? • Interview • Observing • Control group • Opinion survey

30. Finally we agree to ask “Can we afford not to evaluate training?”

31. The Kirkpatrick Training Evaluation Model Thank You