The student’s world: Learning and Evaluation from the Learner’s Perspective

1. The student’s world:Learning and Evaluation from the Learner’sPerspective Kevin W. Eva Centre for Health Education Scholarship University of British Columbia

2. Presenter Disclosure I have been unable to achieve any conflicts of interest for the material included in this presentation

3. I just want to learn something new!!

4. Anover-generalization

5. \ˈfēd-ˌbak\ Providing constructive feedback (1991) • Discuss the basis for evaluation • Base feedback on specific observations • Use a balance of positive and negative • Et cetera …

6. Feedback as Improv “The dialogue, action, story, and characters are created collaboratively as the story unfolds.”

8. The student’s world • Evaluating performance • Incorporating feedback • Building professional identity

9. What do these things have in common? Running a triathlon Writing a prescription Driving a car Offering a consultation

10. Judging one’s own ability “It’s a comfortability … when I don’t have to think about it that’s when I realize, more or less, [that I] have achieved my goal” (Kindler, et al., 2013)

12. Theself-assessment literature • Hundreds of articles • Many literature reviews • One conclusion Self-assessment ability is generally poor

13. Whycan’t we self-assess? • Imagine yourself driving • It’s easy to do … • … but details are inevitably left out • … and those details are crucial to determining the outcome of the actual experience (Vallone, et al., 1990)

14. Whycan’t we self-assess? • Furthermore … • Did anyone imagine themselves having difficulty driving? • When it’s easy to imagine an event we tend to overestimate its likelihood (The availability heuristic: Tversky and Kahneman, 1973)

15. Jowett, etal. (2007) Learner Material

16. Thegeneralpoint • When students self-assess they do so by drawing inferences…. …just as we do when we make judgments about their performance… …and, as a result, they’re just as fallible

17. Implications • External guidance is required …

18. The student’s world • Evaluating performance • Incorporating feedback • Building professional identity

19. Shute (2008) “formative feedback has been shown … to improve students’ learning and enhance teachers’ teaching to the extent that the learners are receptive and the feedback is valid, objective, focused, and clear. [BUT] there is no “best” type of formative feedback for all learners and learning outcomes

20. Shute (2008) …to the extent that the learners are receptive…

21. “Physician agreement with colleagues’ scores correlated positively with overall mean scores from colleagues (R=.48)” Sargeant et al. (2003)

22. (Eva, et al., 2010) Which factors, personal or external, most influence students’ generation of learning goals?

23. What predicted generation of learning goals? Overall R2=0.45 NOTE: Absolute values illustrated for ease of comparison (Eva, et al., 2010)

24. “It is impossible for a man to learn what he thinks he already knows” Epictetus (circa 100AD)

25. Discounting the negative “I mean sometimes you’ll do it and somebody will watch you and it didn’t work and they’re like, ‘You did everything right, it’s just one of those things.’ But if it’s successful then I think, yeah, I can do that.” “Some days I’ll go home and say I was a really good doc today and feel good about what happened in the day. And then other days you know, if I have a headache, maybe I wasn’t so good today.” (Eva, et al. 2011; AHSE)

26. Seeking the positive (Harrison, et al., 2013)

28. Self Efficacy • Influenced by feedback, but also influences future performance • Not merely a passive reflection of performance, but part of a self-fulfilling prophecy that affects performance (see Bandura, 1977)

29. Implications • External guidance is required … … that is sensitive to the perceptions and perspectives of the recipient …

30. The student’s world • Evaluating performance • Incorporating feedback • Building professional identity

31. The need for confidence/safety “If you feel bad and you feel uncomfortable, you’re gonna not have the confidence to talk about mistakes …you don’t get any kind of constructive feedback because …you’re not comfortable saying it” (Eva, et al., 2012)

32. A common perspective Feedback High Self-perception Learning goals Motivation Awareness Benchmarking Performance Low Good Poor

33. A common model in HPE revised Feedback High Self-perception Low Performance Good Poor

34. “Desirable difficulties” Creating situations that will enable learners to discover the limits of their knowledge/ability (see Eva, 2007 (BMJ); 2009 (AHSE))

35. Test-enhanced learning See Larsen, Butler, and Roediger (2008)

36. What predicts review time? (Agrawal, et al., 2012)

37. “Education is learning what you didn’t even know you didn’t know” (Boorstin, 1914)

38. “Studies that augmented feedback with consultation produced substantially larger differences, but other methodological variations had little effect” (Marsh and Roche, 1997)

39. Credibility Of relationships: Engagement and awareness Of the process: Validity and authenticity Of intent: Beneficence and non-maleficence (Sargeant, et al., 2010)

40. Credibility “It doesn’t mean it’s nice to hear it. It’s not so nice; you don’t feel so good after it. But it’s a difference when it’s real, if what they’re saying is just to help you, or if it’s just to put you down” (Eva, et al., 2012)

41. Implications • External guidance is required … … that is sensitive to the perceptions and perspectives of the recipient … … while taking into account the inherent risk/strain learners feel

42. 2 slides of summary • 2 slides of discomfort

43. Promoting receptivity • Self-assessment is vital • Self-confidence is important • Self-assurance is powerful • Self-discovery is crucial • Self-reliance can be detrimental

44. Moving beyond the script Things to do: • Avoid normative comparisons • Use praise wisely • Different styles for different tasks (see Shute, 2008) • Keep the relationship/dialogue in mind

45. Boehler, Rogers, et al. (2006)

46. If you think this is someone else’s problem, you’re not alone … … Pronin, Lin, Ross (2002)

47. Thanks kevin.eva@ubc.ca Further reading: Eva, Regehr, Gruppen (2012). Blinded by insight.