Utility and Validity of Mini-CEX in a Graduate Entry Medical Program

1. Utility and Validity of Mini-CEX in a Graduate Entry Medical Program Suzanne Wilson,Gary Rogers, Simon Broadley Gold Coast Hospital School of Medicine • Introduction: • Assessment of clinical skills is an essential component of any clinical program. • Observed structured clinical examinations (OSCE) have done much to standardise this process and widely adopted. • OSCE often does not involve ‘real’ patients, occurs in a somewhat artificial examination setting and may not truly reflect a student’s performance ‘on the wards’. • The Mini-clinical evaluation exercise (Mini-CEX) has been developed as a flexible clinical assessment tool. • Reliability of Mini-CEX in post-graduate and undergraduate settings have been demonstrated. • Performance against OSCE in an undergraduate setting has not been presented. • The Griffith Medical Program is a 4 year post-graduate course, in which the final 2 years are mainly based around 9 clinical rotations. • We have utilised the Mini-CEX assessment process in these rotations with students being required to complete 2 Mini-CEX assessments in 8 of the rotations. • Students also undertake an 18 station OSCE towards the end of the course. Figure 2. Mean (and 95%CI) Mini-CEX scores by clinical rotation. • Methods: • Data were collected on 70 students completing Years 3 and 4. • A total of 1192 Mini-CEX scores were collated by rotation, examiner and task. • Cronbach’s alpha was used to determine reliability. • Pearson’s correlation coefficient were calculated against OSCE scores as a measure of validity. • Results: • Mini-CEX scores increased only slightly over time in the program (Figure 1). • Scores in disciplines varied but there were no significant differences or trends (Figure 2). • Cronbach’s alpha for Mini-CEX assessments, was 0.607 compared with 0.739 for the OSCE. • The Pearson’s correlation coefficient for Mini-CEX against OSCE performance was significant (p = 0.015) but the correlation was modest 0.291 (Figure 3). • Discussion: • The Mini-CEX process was understood by clinicians and students. • There were Issues around using an originally formative tool for summative assessment. • Some concerns from students in relation to variability of both the process and the application of assessment standards for the Mini-CEX. • Mini-CEX is less reliable than OSCE. • Greater variability in terms of tasks, raters, times and locations. • Correlation between Mini-CEX and OSCE is not high. • Random variation in both assessments. • May be measuring different things. • Mini-CEX is useful and valid in assessing student clinical performance. Figure 3. Mini-CEX score vs. OSCE score. • Purpose/Objectives: • To study the reliability and validity of Mini-CEX in an undergraduate setting. Figure 1. Mean (and 95%CI) Mini-CEX scores over time.