Develop your Evaluation Toolkit Paula Buskard Loyalist College

1. Develop your Evaluation ToolkitPaula BuskardLoyalist College

3. Workshop Learning Outcomes • Explain objectives of “learning outcomes” • List “measureable” verbs when writing LO’s • Identify different methods used to evaluate learning • Explain advantages and concerns of selected evaluation methods • Apply different methods to evaluate meeting LO’s of this workshop

4. Learning Outcomes Exercise • In small groups, write on the top of provided paper a learning outcome. • Learning outcome can be one you are using in a course you are teaching, plan on using for your microteach, made up as a small group.

5. What is Needed to know (Learning Outcome)What is Nice to know(Filler)

6. S – Specific M – Measureable A – Achievable R – Relevant T - Timely

7. Verbs to avoid when writing LO’s • Understand • Enjoy • Know • Realize • Be aware of • Perceive

8. Learning Outcomes Exercise Repeat • Refer to hand-outs. • If needed, in your small group rewrite your learning outcome using “Blooms Taxonomy” or other “action” verbs. • When done, post your learning outcome on the wall

9. Evaluation – What is it??

10. Formative: • diagnostic tool (Where are learners at? What have they got so far? Do I need to go over something again?) • help learners take stock of and assess their own progress towards learning goals • Summative: • assess a learners achievement of meeting the learning outcomes • determine grades

11. Evaluation Methods Snowball Exercise • Each group move to a posted Learning Outcome (does not have to be yours). • Take 2 minutes (I will call time) to identify evaluation methods that could be used to measure learning of the LO. • When time is called, move to the next LO and repeat. • Once each group has had the opportunity to identify evaluation methods for each Learning Outcome let me know. • Share with large group.

12. EVALUATION METHODS Multiple Choice True/False Short Answer Essay Problem Solving in Situations Structured Oral Test One to One Dialogue Panel Interview

13. Evaluation Methods (cont) • Simulation Case Study • Work Sample • On Site Visit • Unobtrusive Observation • Essays/Reports/Diaries/Logs/ Journals/Blueprints/Art/Photography/Books/Articles • Portfolios

14. Evaluation Methods- Advantages and Concerns

15. Advantages/Concerns Snowball Exercise • In your small group go to a posted evaluation method. • Identify an advantage of using that particular method to evaluate learning. • Identify a concern to using that particular method to evaluate learning. • After 2 minute (I will keep time), move to the next Evaluation Method. • Once your group has circulated around to all posted Evaluation Methods let me know. • Share with large group.

16. Hand out of Evaluation MethodsAny thoughts??

17. Test planning

18. What do you want your learners to know?

19. Four Points to Consider: 1 – Measure what is most important rather than what is easiest to measure 2 - Be aware of the learning embedded in evaluation 3 – Understand the evaluator’s power 4 – Take care not to reduce the curriculum to only what is evaluated

20. Guidelines for planning • Start early (holidays, sequence, scaffolding) • Determine the number of items and time limitations • Construct the question- make sure that it measures learning and is linked to content objectives/outcomes • Edit the questions for errors in spelling, sentence structure, etc.

21. Guidelines for planning (cont) Group the questions according to type and topic (make easier ones first) Write test directions (include marking scheme) Have the answers “ready” for post test debrief Review test

22. Preparing your Learners

23. No Surprises – Learners Know What to Expect • the content (knowledge and skills being tested) • the timing (give warnings) • the type of evaluation activity being used (multiple choice, short answer, role-play, combination, etc.) • criteria on which their performance will be judged (embedded in the Learning Outcomes)

24. debriefing and feedback

25. Teacher Debrief Learners Debrief Exercise: In your small groups answer these 2 questions: What could teachers consider after a test? What could teachers do to help their learners after a test? W

26. Teacher Debrief: • Did my learners understand what was asked on the test? • Do the results indicate that the learners were prepared for the activity? • Are the results consistent with other evaluations? • If I use this procedure again, what changes will I make? • What other methods of evaluation might be more appropriate? • Were there questions that the learners missed that were the fault of the teaching? How will I remedy this? • Did the test differentiate between the learners who prepared and those who did not? • Is there evidence of cheating? If so, what can I do about it (this time and in the future?)

27. Learners Debrief: • Quickly go over the test and provide the correct answers to all of the questions. • Discuss the outstanding questions – those that most students answered well, those that some may have got stuck on, etc. • Open the floor for discussions. • After a reasonable time is spent on a class discussion, invite students to make appointments with you to discuss individual concerns.

29. REFERENCES AND RESOURCES: Andrus, C and Bloor, D (2005) Teaching tools Gross-Davis, B. (2001). Tools for teaching. San Francisco: Jossey-Bass Fenwick T. and Parsons, J. (2000). The art of evaluation: A handbook for educators and trainers. Toronto: Thompson Educational Publishing http://www.naacls.org/docs/announcement/writing-objectives.pdf http://rubistar.4teachers.org/index.php http://pareonline.net/getvn.asp?v=7&n=3