Download
1 / 16
160 likes | 174 Views
This training unit provides an overview of how to evaluate teaching and learning, develop evaluation strategies, and use appropriate evaluation tools. Learn how to assess if learning outcomes have been achieved, determine the value of a learning session, and improve future teaching. Explore different evaluation models to measure impact, application, learning, and reaction. Understand stakeholder consultation, information needs, evaluation approaches, and dissemination of findings. Tailor your evaluation to meet the needs of your learning program.
E N D
Training Trainers and EducatorsUnit 8 – How to Evaluate Aim • To provide an overview of how to effectively evaluate teaching and learning Learning outcomes • Develop evaluation strategies for their teaching and learning activities • Use a range of evaluation tools appropriate to their teaching and learning activities
Evaluation ‘Evaluation is the collection, analysis and interpretation of information about any aspect of a programme of education or training, as part of a recognised process, judging its effectiveness, its efficiency and any other outcomes’. Thorpe (1988)
Why do we evaluate? • To identify if aims and learning outcomes have been achieved • To determine the value of the learning session • To determine what worked well and what could be improved • To demonstrate the impact of the learning in relation to knowledge, performance and service improvement
Checklist(www.test4best.scot.nhs.uk) Evaluating the impact of a new educational solution can be complex, as a wide range of factors may need to be considered Outcomes and outputs • Think about the original analysis of the issue which prompted the learning programme to be developed. • Think about the links between • the issue • the outcome i.e. the anticipated change in behaviour, quality, performance and efficiency of a service or individual • the output i.e. the activity developed to achieve the outcome
Issue • There has been an increase in complaints about the way telephone calls to a clinic are handled Outcome • Elimination of complaints about call handling Output • Telephone skills training for staff who take and make calls How will output address outcome? • Clearer frameworks for call handling will help staff deal more effectively with routine calls • Improved skills will help staff deal with more complex difficult calls • Reviewing the links between the issue, the outcome and the outputs
Stakeholder consultation • What do they want to find out from the evaluation ? • This will help establish and agree the aims of any evaluation of impact Information needs • Do you have information which will act as a baseline against which you will measure development ? • If you don’t have this information, what effect will it have on the aims of your evaluation? • Checklist(www.test4best.scot.nhs.uk )
Approaches to evaluation • What type of information is needed ? • Do you need small scale evaluation (self assessment, user feedback) or likely to require external evaluation ( strategic importance) • How will the information be gathered ? • Frequency and timing of evaluation ( one evaluation or a series of evaluations) Dissemination of findings • Who needs to know about the findings of this evaluation ? • How will you disseminate the evaluation report ? • Checklist(www.test4best.scot.nhs.uk )
Who is the evaluation for? What do you need to know? Why do you need to know it? Will the evaluation make a difference? How do we evaluate?
Level 4 Impact – the impact of learning on practice in terms of results within the organisation • Kirkpatrick’s Four-level approach(1959) Level 3 Application – change in job behaviour and transfer of learning to the workplace Level 2 Learning - the resulting increase in knowledge or capability before and after training Level 1 Reaction - what they thought and felt about the learning programme
Kirkpatrick’s Four level approach examples Level 4 Follow up participants and managers ,productivity data look at costs, quality, patient experience Level 3 Observation, reflective log by participant, CPD evidence Level 2 Pre and post course knowledge questionnaire, Self assessment of knowledge, observation of performance, Level 1 Reaction questionnaires, post its
Evaluation models:Hamblin’s five - level approach ( 1974) • Level 1 Reactions • Level 2 Learning • Level 3 Job behaviour • Level 4 Organisation – the effects on the organisation, from participant’s job to performance changes • Level 5 Ultimate value – the financial effects, both on the organisation and the economy Hamblin AC (1974) Evaluation and Control of Training, Maidenhead, McGraw Hill.
Think about a learning programme you want/are delivering • Worksheet 1 • Stakeholders • Information needs • Evaluation approach • Dissemination • Worksheet 2 • Evaluation strategy • Group work
Evaluation is an essential component of any teaching and learning programme • Evaluation needs to be considered carefully at the start of any learning programme development and well before implementation. • There is no right or wrong way to evaluate • Remember • Who is the evaluation for? • What do you need to know? • Why do you need to know it? • Will the evaluation make a difference? • Summary
