Self-assessment, Peer Review & Feedback

1. Self-assessment, Peer Review & Feedback • Testing Design Work Against • Standards

2. Self-Assessment & Self-Adjustment

3. The research jibes with this common sense: • To achieve any goal you must learn how to self-assess your learning and your performance • One of 3 chief findings in How People Learn • Success depends upon valid self-assessment and timely self-adjustment on your own you with minimal outside prompting and close supervision

4. From the research: one of 3 chief findings - • “The teaching of metacognitive skills should be integrated into the curriculum. Because metacognition often takes the form of an internal dialogue, many learners may be unaware of its importance unless the processes are explicitly emphasized by teachers. • “Research has demonstrated that learners can be taught these strategies, including the ability to predict outcomes, explain to oneself, note failures to understand, activate background knowledge, plan ahead, and apportion time and memory… • How People Learn, p. 14, 21

5. Research on metacognition, part 2 • The model for using the meta-cognitive strategies is provided initially by the teacher, and learners practice and discuss the strategies as they learn to use them. • Ultimately, learners are able to prompt themselves and monitor their own comprehension without teacher support. [transfer] • How People Learn, pp. 18-19

6. So, what follows for staff? • Models, feedback, adjustments, feedback • This cycle has to be woven into professional development and team meetings • This is the essence of LessonStudy, the highly successful Japanese approach • Depends on model units, ubd standards, a self-assessment protocol, and practice in self-assessment after having it modeled

7. Unit Design Cycles Backward Design: • Stage 1 • Stage 2 • Stage 3 based on: •Program goals • Performance gaps Designer’s observation Student feedback Trial Design Feedback Working smarter via: • design teams Expert review Reviewed against Design Standards by: • self • peers Analysis of student work 7

9. UbD – Design Standards • See the design standards and review materials in the UbD Workbook: • design standards linked to the three stages ‘backward design’ • each standard contains criteria, a three-point performance scale and indicators • serve as guides for peer & external reviews

10. Peer Review: Goals • Provide specific feedback and guidance to designer based on design standards. • Improve the quality of unit designs. • Increase understanding of the qualities of effective curricular design. • Develop a more collaborative and open culture

11. Peer Review Process • Step 1 - Overview of unit • Designer presents a brief overview of unit and requests specific feedback from the group. • Designer leaves the group. Suggestion:Designer should take another unit to review while they are waiting.

12. Peer Review Process • Step 2 - Individual reviews of unit • designs without designer present • Establish group roles. • Reviewers individually (and silently) review and evaluate the design based on review criteria and record comments about strengths and weaknesses on the Individual Review Form.

13. Peer Review Process • Group roles: • Timekeeper/Facilitator • Recorder • Spokesperson(s)

14. Peer Review Process • Step 2 - Individual reviews of unit • designs without designer present • Establish group roles. • Reviewers individually (and silently) review and evaluate the design based on review criteria and record comments about strengths and weaknesses on the Individual Review Form.

15. Peer Review Process • Step 3/4 - Review group discussion • without designer present • Review group discusses the design and records key questions, feedback and guidance on the Group Review Form. • Spokesperson(s) verbally summarizes the group’s comments.

16. Peer Review Process • Step 5 - Group discussion of unit with designer • Reviewers present feedback and guidance to the designer. • Designer listens, takes notes, and asks clarifying questions.

17. Feedback Feedbackis descriptive. It provides commentary on the extent to which the design meets the stated goals (intent vs. effect). e.g., The first performance task does not seem to align with any of the unit Understandings.

18. Feedback, defined • Useful information on the actual against the optimal;the intent vs. effect • Feedback reports back on what you did, against a specific target—no personal value or aesthetic judgment is made • Feedback is descriptive, not evaluative • Feedback is not praise or blame • Feedback is not guidance/advice

19. Examples of feedback about ubd • “That question seems to me to be the most ‘essential’ question - it is thought-provoking and can profitably be asked and re-asked throughout the unit .” • “That second one is really more of a leading factual question than an essential question.” • “I cannot find any assessment of your 2nd understanding” • “Those activities align nicely with Understandings 1 and 2. But I don’t see any activities directly related to #3.” • “I think a student could do well on that performance assessment without really understanding understanding #2 (which you say it addresses).”

20. Feedback • The most useful feedback is: • specific • guided by criteria • understandable to the receiver • timely © 2000 Grant Wiggins & Jay McTighe

21. Feedback vs. Guidance Guidanceoffers suggestions for improving the design, based on the designer’s stated goals (i.e., how to narrow the gap between intent & effect). e.g., Perhaps a different Understanding would be more closely aligned with your first assessment task. © 2000 Grant Wiggins & Jay McTighe

22. Don’t Confuse Feedback with Guidance (Advice) • Too many people jump to guidance without first providing feedback and making sure the performer understands it and agrees. • Feedback: what you did or did not do, given a standard; a neutral description of your performance or product • Guidance: what you might do to honor the feedback - good advice

23. F G N KEY: F = Feedback G = Guidance N= Neither Check for Understanding 1. What a great idea! 4 2. The first Essential Question is factual in nature (recall only). 4 3. Perhaps you could begin with an experiential activity as a ‘hook’. 4 4. I never use Word Searches in my classroom. 4 © 2000 Grant Wiggins & Jay McTighe

24. F G N KEY: F = Feedback G = Guidance N= Neither Check for Understanding 5. The mystery game is likely to ‘hook’ and engage the students. 4 6. We didn’t like the culminating activity. 4 7. You might try using the ‘concept attainment’ technique here. 4 8. We didn’t see any assessment of the second understanding. 4 © 2000 Grant Wiggins & Jay McTighe

25. Tip for Effective Peer Review • Remember that the primary goal of peer review is to provide useful feedback and guidance to help improve the design.

26. Tips for Effective Peer Review • Reviewer’s comments should be specific and “objective” - i.e. guided by and made in reference to the design standards, not personal taste.

27. Tips for Effective Peer Review • Try to understand the design and its intent before offering feedback and guidance. The designer/teacher should feel that you are trying to improve their design, not substitute your own goals or methods for theirs.

28. Feedback and its power

29. feedback and its use is key to great gains • Black & Wiliam meta-analysis: • “There is a body of firm evidence that formative assessment is essential... We know of no other way of raising standards for which such a strong prima facie case can be made.” • Black and Wiliam (1998) “Inside the Black Box: Raising Standards through Classroom Assessment,”Phi Delta Kappan, volume 80, 2 (October), pp. 139 ff. • Cf. Working Inside the Black Box: Assessment for Learning in the Classroom,by Paul Black, Christine Harrison, Clare Lee, Bethan Marshall, and Dylan Wiliam Phi Delta Kappan, Volume 86, #1 (September, 2004)

30. Feedback: Harvard’s “most effective” courses • from Making the Most of College: • "The big point—it comes up over and over again as crucial—is the importance of quick and detailed feedback. Students overwhelmingly report that the single most important ingredient for making a course effective is getting rapid response on assignments and quizzes. • "Students suggest that it should be possible in certain courses to get immediate feedback. They suggest that the professor should hand out an example of an excellent answer. - Richard Light

31. Feedback as key (cont.): • "Secondly... an overwhelming majority are convinced that their best learning takes place when they have a chance to submit an early version of their work, get detailed feedback and criticism, and then hand in a final revised version... • Many students observe that their most memorable learning experiences have come from courses where such opportunities are routine policy."

32. Feedback is vital for faculty, too • “Faculty members at Harvard were asked what single change most improved their teaching. Two ideas swamped all others. One is enhancing student awareness of the big picture, ‘the big point of it all’. The second is the importance of helpful and regular feedback from students so a professor can make midcourse corrections.” • - Harvard Assessment Seminar, 1993

33. Feedback Depends upon Models • based on: • Exemplars/Models • Expert Commentary on Models and non-models • Ideal specifications or performance standards

34. No gains = poor feedback system • No imrpovement over time show the problem is not “prof dvlmpt” or “staff” but the feedback system • Ponder: how would a school’s Cross Country team do if we only kept the place of finish and graded you on your relative place of finish - and did not keep your times?

35. Feedback/Self-assessment System - Healthy • Performers seek feedback on their own and know that it is in their interest - even if the news is bad • Performance improves at all levels; there is obvious “value added” • Improved performance occurs more rapidly than is typical or expected • Few quarrels about the feedback (if there are, it is about the meaning of the results) • Feedback use opportunities are central to the job • Norms and standards rise over time: what was once considered extraordinary performance becomes more common

36. Feedback/Self-assessment System - Unhealthy • Learners fear, resist, do not seek, or ignore feedback • Learner performance rarely improves much beyond what is typical • Novices struggle to improve; they do not know “what you want.” Their self-assessment is very inaccurate • Many quarrels about the credibility and meaning of the results; anecdotes and effort trusted more than anything • Training is too prone to coverage and activities, with little opportunity to get feedback and use it repeatedly • Norms stay the same, standards rise - and expectations are thus lowered

37. Excellent feedback • Some criteria: • Timely • user-friendly - in approach and amount • Descriptive & specific re: performance • Consistent • Expert • Accurate • Honest, yet constructive • Derived from concrete standards • On-going

38. So, what follows for staff? • How can we make feedback more common, useful, credible - and timely? • Sample their work regularly • Peer review • Use online resources (e.g. ubdexchange)