160 likes | 247 Views
Evaluate Strategic P rogram S uccess. Changing Mindsets Conference August 2010 – TAFE Development Centre. Michelle Harrex Senior Teaching and Learning Co-ordinator Professional Learning and Development. Objectives of this session.
E N D
Evaluate Strategic Program Success Changing Mindsets Conference August 2010 – TAFE Development Centre
Michelle Harrex Senior Teaching and Learning Co-ordinator Professional Learning and Development
Objectives of this session • Knowledge of issues to consider when evaluating teaching/ course/ program • Abilities to plan evaluation of your practices in a course/ program
Will the information add value? Objective: A path to value
Two examples of evaluation approaches • Action Research for ongoing improvement of quality (Formative Evaluation) • Pre-Post Test for measuring change, adapted from experimental/scientific approaches that use Treatment and Control groups (Summative evaluation)
Is the information valuable? Evaluate to meet stakeholder needs! In the current climate what is considered valuable evaluation data? Learner engagement vs business imperative (the tug of war) Where do you stand, where does your organisation stand, where do your clients stand?
Define what is valuable then develop a sound evaluation framework? How to evaluate? (Key considerations) • The evaluation model depends on the purpose of evaluation (fitness for purpose). • Data/information can be qualitative and/or quantitative. • Ideally, data should be from many sources to reduce biases (increase validity and reliability). • Data analysis and conclusions should be appropriate.
The structure of a Training Evaluation Problem – a model 1 2 3 4 5 TRAINING WORKPLACE Reactions Results Learning Behaviour
Task : Consider a course/ program you are teaching/developing. For each purpose of evaluation, identify the following: • Major focus of the evaluation • Typical issues that should be explored, i.e. the types of questions to ask • Types of evidence that should/can be collected
Emphasis on results…see handout • Kirkpatrick says we need to progress through the layers…do you agree?
Kirkpatrick Model for evaluating effectiveness of training programs What organisational benefits resulted from the training? To what extent did participants change their behaviour back in the workplace as a result of the training? To what extent did participants improve knowledge and skills and change attitudes as a result of the training? How did participants react to the program?
The benefits of valuable evaluation • Program viability • Customer satisfaction • Competitive edge • Compelling evaluation
Resources Online resources for Teaching/Program Evaluation LTDI, Heriot Watt University (n.d.). Evaluation Cookbook -A practical guide to evaluation methods for lecturers. Retrieved from http://www.icbl.hw.ac.uk/ltdi/cookbook/contents.html McNamara, C. (n.d.). Basic Guide to Program Evaluation. Retrieved from http://www.managementhelp.org/evaluatn/fnl_eval.htm Penn State University (n.d.). Program Evaluation. Retrieved from http://extension.psu.edu/evaluation/ TDA (2007). Impact Evaluation of Continuing Professional Development. Retrieved from http://www.tda.gov.uk/upload/resources/pdf/i/impact_evaluation.pdf The University of Sheffield (2008). LeTSEvaluation Resources –Impact Evaluation. Retrieved from http://www.sheffield.ac.uk/lets-evaluate/impact