Download
1 / 26
270 likes | 425 Views
Super*Vision Training Program. Evaluation Methodology. Jing Wan. Components. Introduction Logic model Evaluation methods(Design, data collection, analysis) Communication & dissemination. Introduction. Evaluation questions Formative evaluation Summative evaluation
E N D
Super*Vision Training Program Evaluation Methodology Jing Wan
Components • Introduction • Logic model • Evaluation methods(Design, data collection, analysis) • Communication & dissemination
Introduction • Evaluation questions Formative evaluation Summative evaluation • Data collection methods & Measures Formative evaluation Summative evaluation (Qualitative measures+ quantitative measure) • Communication & Dissemination
Program Analysis • Overview of logic model
Program Analysis • Inputs: Training structure & set-up Training process Training content Trainer Trainee Supportive environment
Program Analysis • Outputs & Outcomes Program products & activities Changes in trainees’ KSA Improved workforce & organizational performance
Evaluation Methods • Evaluation questions • Evaluation design • Data collection methods
Evaluation questions • Related to inputs Is training content relevant to the trainees’ job assignment? How satisfied are the trainees with their involvement in the training session? • Related to outputs Does the training program provide the planned assignments?
EQ • Related to outcomes/impacts Did the trainees experience any changes in their KSA, and job performance? What are the effects of the training program on each department of DHSMV (eg. Improved work efficiency, enhanced networking, and partnerships)?
Evaluation Design • Formative (process) evaluation design How well the training program matches the evaluation theory behind its design, what the program actually does during the process. • Summative (outcome) evaluation design How the training program is related to changes in the trainees, whether the program causes the desired changes.
Data collection methods • Formative (process) evaluation Quantitative measure: What to measure: trainees training experience How to measure: after training survey From whom collected: trainees How to analyze: statistical analysis for quantitative data
Data collection Cont. • Formative (process) evaluation Qualitative measure: What to measure: the training meets the trainees’ expectation How to measure: after training survey From whom collected: trainees How to analyze: content analysis for qualitative measure
Data collection cont. • Summative (outcome) evaluation Qualitative & quantitative measures What to measure: short-term outcomes How to measure: follow-up questionnaire & telephone interview From whom collected: trainees & subordinates How to analyze: statistical analysis & content analysis
Communication & Dissemination • Communication challenges General evaluation anxiety (results can affect decisions) Disengagement Management operating style (resistant to change, rapid staff turnover, dysfunctional information-sharing system)
CD cont. • Communication methods Written reporting • Final evaluation report • Progress reports Verbal presentations • Debriefing meetings Bring together stakeholders to present key evaluation findings and recommendations or other evaluation components.
