1 / 16

Training Trainers and Educators Unit 8 – How to Evaluate

This training unit provides an overview of how to effectively evaluate teaching and learning activities, develop evaluation strategies, and use appropriate evaluation tools.

johnsonkelly
Download Presentation

Training Trainers and Educators Unit 8 – How to Evaluate

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Training Trainers and EducatorsUnit 8 – How to Evaluate • Aim • To provide an overview of how to effectively evaluate teaching and learning • Learning outcomes • Develop evaluation strategies for their teaching and learning activities • Use a range of evaluation tools appropriate to their teaching and learning activities

  2. Evaluation ‘Evaluation is the collection, analysis and interpretation of information about any aspect of a programme of education or training, as part of a recognised process, judging its effectiveness, its efficiency and any other outcomes’. Thorpe (1988)

  3. Why do we evaluate? • To identify if aims and learning outcomes have been achieved • To determine the value of the learning session • To determine what worked well and what could be improved • To demonstrate the impact of the learning in relation to knowledge, performance and service improvement

  4. Testing for the Bestwww.test4best.scot.nhs.uk

  5. Checklist(www.test4best.scot.nhs.uk) • Evaluating the impact of a new educational solution can be complex, as a wide range of factors may need to be considered • Outcomes and outputs • Think about the original analysis of the issue which prompted the learning programme to be developed. • Think about the links between • the issue • the outcome i.e. the anticipated change in behaviour, quality, performance and efficiency of a service or individual • the output i.e. the activity developed to achieve the outcome

  6. Issue • There has been an increase in complaints about the way telephone calls to a clinic are handled Outcome • Elimination of complaints about call handling Output • Telephone skills training for staff who take and make calls How will output address outcome? • Clearer frameworks for call handling will help staff deal more effectively with routine calls • Improved skills will help staff deal with more complex difficult calls • Reviewing the links between the issue, the outcome and the outputs

  7. Stakeholder consultation • What do they want to find out from the evaluation ? • This will help establish and agree the aims of any evaluation of impact Information needs • Do you have information which will act as a baseline against which you will measure development ? • If you don’t have this information, what effect will it have on the aims of your evaluation? • Checklist(www.test4best.scot.nhs.uk )

  8. Approaches to evaluation • What type of information is needed ? • Do you need small scale evaluation (self assessment, user feedback) or likely to require external evaluation ( strategic importance) • How will the information be gathered ? • Frequency and timing of evaluation ( one evaluation or a series of evaluations) Dissemination of findings • Who needs to know about the findings of this evaluation ? • How will you disseminate the evaluation report ? • Checklist(www.test4best.scot.nhs.uk )

  9. Who is the evaluation for? What do you need to know? Why do you need to know it? Will the evaluation make a difference? How do we evaluate ?

  10. Evaluation Models

  11. Level 4 Impact – the impact of learning on practice in terms of results within the organisation • Kirkpatrick’s Four-level approach(1959) Level 3 Application – change in job behaviour and transfer of learning to the workplace Level 2 Learning - the resulting increase in knowledge or capability before and after training Level 1 Reaction - what they thought and felt about the learning programme

  12. Kirkpatrick’s Four level approach examples Level 4 Follow up participants and managers ,productivity data look at costs, quality, patient experience Level 3 Observation, reflective log by participant, CPD evidence Level 2 Pre and post course knowledge questionnaire, Self assessment of knowledge, observation of performance, Level 1 Reaction questionnaires, post its

  13. Learning Programme-Mentor Update Training for Nurses

  14. Evaluation models: Hamblin’s five - level approach ( 1974) • Level 1 Reactions • Level 2 Learning • Level 3 Job behaviour • Level 4 Organisation – the effects on the organisation, from participant’s job to • performance changes • Level 5 Ultimate value – the financial effects, both • on the organisation and the economy • Hamblin AC (1974) Evaluation and Control of Training, Maidenhead, McGraw Hill.

  15. Think about a learning programme you want/are delivering • Worksheet 1 • Stakeholders • Information needs • Evaluation approach • Dissemination • Worksheet 2 • Evaluation strategy • Group work

  16. Evaluation is an essential component of any teaching and learning programme • Evaluation needs to be considered carefully at the start of any learning programme development and well before implementation. • There is no right or wrong way to evaluate • Remember • Who is the evaluation for ? • What do you need to know ? • Why do you need to know it ? • Will the evaluation make a difference ? • Summary

More Related