Student experiences of assessment in two problem-based dental curricula: Adelaide and Dublin

Student experiences of assessment in two problem-based dental curricula: Adelaide and Dublin Tracey Winning Elaine Lim Grant Townsend Dental School, The University of Adelaide

Introduction • PBL in dental education with matched assessment methods • Assessment drives student learning: • Student perceptions of test demand influences approaches / outcomes (Biggs 1996) • May/may not match intended objectives “Not what teacher believes assessment is testing which governs students’ behaviour, but their own perceptions” (Boud, 1990)

Aims To explore influence of assessment on learning within assessment monitoring processes, we need open dialogue with students' about their perspectives of assessment experiences. • How do studentsrespond to specific assessment experiences that they identify and what did they gain/learn from these experiences? • Which purposes, features and methods of assessment students rate as important?

Sites and Participants • Sites:Adelaide & Trinity: PBL for more than 5 y • Curriculum features: • Similar objectives and assessment • Trinity: academic results; 98% school leavers; integrated • Adelaide: combination of criteria; approx 25% tertiary transfer; hybrid • Participants: • 3rd year and 5th year volunteers n=17 n=18 n=25 n=25 Adelaide Trinity Adelaide Trinity

Data Collection and Analysis • Survey: • Critical incident: identify experience, context, thinking/feeling • Likert scale items: rate importance of assessment purposes, features associated with positive outcomes and methods in assessing learning • During class, no demographic data, anonymous • Analysis: • Critical Incidents: Codes and themes derived and cross checked • Likert scale items: frequency distributions, rank order, differences between groups: Kruskall-Wallis, p<0.01

Critical Assessment Issues • Methods and experiences: • Majority written and clinic assessments • 1/3 positive/2/3 negative experiences • Issues/Themes: • Grading and marks: unexpected grades, lack of standardisation between assessors • Tutor interactions and Feedback: • Positive: encouraging, supportive, challenging, fair feedback: timely, constructive, specific, strategies • Negative: intimidating/discouraging, indifferent feedback: none or limited, disappointing, demotivating • Assessment Stress: memorable feature; external (related to timing) and student related factors (unfamiliar/unexpected Q/topics) • Assessment organisation and expectations: preparation time, format, content

Purposes of Assessment p>0.01

Features associated with positive outcomes p>0.01

Assessment Methods p>0.01 • 92% 5th y Trinity rated vivas as very/important • cf 58% others (p<0.01)

Discussion • 3rd and 5th year students had similar experiences and perceptions • Some had positive experiences and perceptions of assessment • Majority had negative outcomes (despite attempts to match learning and assessment and support students) • Concern re grades received  activities to clarify and apply assessment criteria and standards (as well as tutors)

Discussion • Tutor feedback on learning highly valued  ie immediate and constructive feedback vs generic or no feedback with little explanation. • Students perceived assessment info to be ineffective  more guidance needed, with interactive discussion throughout year. • PBL scenario-based examinations perceived by majority as providing most important method for judging their learning  learning assessed is consistent with future practice. • Students’ perspectives on the importance of various characteristics of assessment in supporting learning is consistent with good assessment practice

Conclusions • Anecdotal reports confirmed • Common issues were identified ie students at both sites and in literature • Further improvements are needed, eg, student and tutor training in application of assessment criteria • Wide range of student goals, previous experiences and values no course can present an ‘ideal’ assessment scheme but need to monitor students experiences

Acknowledgements • Australian Dental Research Foundation: for Undergraduate Vacation Research Scholarship to Elaine Lim • Dr Mary Kelly, Trinity College, Dublin: for assistance during Elaine’s visit • Students at Trinity College and Adelaide: for their participation

Questions • What approaches have others used to help academic staff and part-time tutors improve their assessment and learning approaches, particularly in clinical settings? • What to others do about informing and checking students assessment expectations? • What can research tell us about the sue of ungraded assessment and the implications of this on student learning processes and outcomes, experience of assessment and future employment and post-graduate opportunities?

Student experiences of assessment in two problem-based dental curricula: Adelaide and Dublin