Does Self Grading Improve Student Assessment Outcomes?

1. Does Self Grading Improve Student Assessment Outcomes? Dawn Schocken, Frederick Slone, Stephen Charles, Alicia Monroe

2. Does Self Grading Improve Student Assessment Outcomes? Background: • Clinical competence is multidimensional with many different methods used for assessments. • The Clinical Performance Examination (CPX) is a collection of twelve 15 minute case simulations used to evaluate the competence in the diagnosis and treatment of patients.

3. Does Self Grading Improve Student Assessment Outcomes? Background: • The CPX is similar in style and nature to the USMLE (United States Medical Licensing Examination) Clinical Skills Step 2 exam that each medical student in the United States must pass for licensure. • The exam is designed to assess student’s communication skills, history taking and physical exam skills and critical reasoning skills.

4. Does Self Grading Improve Student Assessment Outcomes? Background: • Each student is assigned to attend a final examination at the end of their clinical clerkship year. • The CPX represents a six hour, twelve station case assessment experience. • At each station, the student completes both a fifteen minute interactive patient experience and a fifteen minute written assessment.

5. Does Self Grading Improve Student Assessment Outcomes? Summary of work: • The students self-grade their exam as a formative assessment to prepare for the USMLE. • A total of 110 MS IIIs at the end of their third year participated in this assessment requirement. • Each student received an individualized e-mail with their written exams and the grading rubric. A grade sheet was provided. • The e-mail was password protected to prevent either copying the rubric or posting it for general use.

6. Does Self Grading Improve Student Assessment Outcomes? Summary of work: • Following completion of the six hour exam, the students were given an overview of how to complete their self-assessments. • Review of each case rubric was completed to assure that each student understood how to read the case rubrics and score their assessment. • A specific time table was developed to track when the students received their cases and when they returned them graded.

7. Does Self Grading Improve Student Assessment Outcomes? Summary of work: • 110 students returned their self-graded scores. • Each student’s exams were compared to an independently scored exam to validate student scores. • A questionnaire was distributed to determine the student’s perception of this experience. • A comparison was made to determine student scores on the USMLE exam over the past three years.

8. Does Self Grading Improve Student Assessment Outcomes? Summary of work: • The average student reported score on the CPX exam was 79.5%. • The average faculty graded student score was 80.7% on this exam. • The students were on target with the faculty graded scores and appeared to be more critical of their performance.

9. Does Self Grading Improve Student Assessment Outcomes? Conclusions: • To date, all students have successfully passed the USMLE CS Step 2. • Each student provided a self-report they felt more confident and better prepared. • Previous experience with the CS Step 2 demonstrated 94% were successful in their first attempt to pass. • Formative evaluations appear more effective in preparing students to successfully pass the national exam.

10. Does Self Grading Improve Student Assessment Outcomes? Conclusions: • Independent faculty graders provided assurance that the students scores reflected what they actually did. • There was no statistically significant difference between the student’s self scores and the faculty scores when compared. • Students appeared more critical of their work than the independent graders.

11. Does Self Grading Improve Student Assessment Outcomes? Take-home messages: • Formative assessments for MS IIIs more effectively prepared them for the USMLE National CS Step 2 Exam. • A time frame needs to be given to the students for timely return of their scored exams. • An independent team of faculty graders need to assess a certain percentage of these assessments to assure reliability in exam scoring.

12. Does Self Grading Improve Student Assessment Outcomes? Reference Slides Follow

13. Does Self Grading Improve Student Assessment Outcomes? Appendix Slides: • CPX Grading Rubric • CPX Self Graded Score Sheet • Student Questionnaire

14. Does Self Grading Improve Student Assessment Outcomes?

15. Does Self Grading Improve Student Assessment Outcomes?

16. Does Self Grading Improve Student Assessment Outcomes? Comprehensive Clinical Performance Examination Survey: The Comprehensive Clinical Performance Exam (CCPX) consisted of twelve case stations involving multiple patient (SP) encounters. You were assigned a task(s) at each station. The format of this exam was very similar to the Step 2 CS that you will be required to pass for licensure. Both exams have been developed to measure your knowledge, skills and attitudes as they directly relate to communications skills, patient care and clinical reasoning. We are asking you to respond to this questionnaire to give us feedback on the formative CCPX that you have just taken. • I completed my CCPX exam following my last clinical clerkship: ___ Yes ___ No • I received my CCPX and the grading rubrics in a timely manner: ___ Yes ___ No • I was able to open the grading rubric from my CCPX with no problem: ___ Yes ___ No • I was able to complete the grading of my CCPX with no problem: ___ Yes ___ No • I was able to turn in my grading rubric by the deadline assigned to our group: ___Yes ___ No • I felt the CCPX has prepared me to take the USMLE CS Step 2 Exam: ___ Yes ___ No • I have taken the CCPX exam: ___ Yes ___ No