1 / 14

St George’s Advanced Assessment Course thoughts on a final medical assessment – the Short Case

St George’s Advanced Assessment Course thoughts on a final medical assessment – the Short Case . GAME 5 th December 2012 Yvonne Finn. Briefly........... . The Short Cases examination Validity and quality assurance Constructs, domains & descriptors Standard setting

kevlyn
Download Presentation

St George’s Advanced Assessment Course thoughts on a final medical assessment – the Short Case

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. St George’s Advanced Assessment Course thoughts on a final medical assessment – the Short Case GAME 5th December 2012 Yvonne Finn

  2. Briefly........... • The Short Cases examination • Validity and quality assurance • Constructs, domains & descriptors • Standard setting • Pros and cons of whole process

  3. 5MB Short case • 30% of 5MB year assessments • Requirement to pass • OSCE like in set-up • 6 stations; 1 pair of examiners at each • No set script • Usually clinical skills, diagnosis, interpretation • A single GRS

  4. Short case marking sheet • YEAR 5 FINAL CLINICAL EXAMINATIONS 2011/2012 • SHORT CASES • STUDENT NAME:__________________________________________ • STUDENT ID NUMBER:_____________________________________ • DAY: 1 - WEDNESDAY 2 - THURSDAY (PLEASE CIRCLE) • SESSION: MORNING AFTERNOON (PLEASE CIRCLE) • STATION NUMBER (1 – 6): _________________ • AGREED GLOBAL MARK (OUT OF 10, to the nearest 0.5)*:_________ • COMMENTS (if any): • EXAMINERS’ SIGNATURES: _______________________ _____________________ *Please consider the following elements of the student’s performance in deciding upon his/her mark for this station: communication skills, approach to physical examination, ability to elicit abnormal physical signs, ability to interpret physical findings. Please use the full range of marks from 0 to 10 (to the nearest 0.5), referring to the following broad grade brackets as a guide: 0-4 = CLEAR FAIL; 4.5 = BORDERLINE PASS/FAIL; 5/5.5 = PASS; 6/6.5 = SECOND CLASS HONOURS; 7-10 = FIRST CLASS HONOURS. The gold medal viva candidate is likely to score well in excess of 7/10 in most stations.

  5. Validity • “Validity is the sine qua non of assessment , as without evidence of validity , assessments in medical education have little or no intrinsic meaning”(1) • Contemporary view of validity: a unitary concept, with construct validity as the whole of validity • Purpose of validity testing is to support (or refute) the proposed interpretations of the test scores. 1. Downing S Med Ed 2003; 37: 830-837

  6. Validity in Clinical Assessment • “Validity depends on the fidelity of the simulation” • Validity depends on the implementation of the assessment” Van derVleuten, St Georges; Advanced Assessment Course

  7. Validity testing by... • Map test content to learning objectives of modules • Content representativeness of test material • Peer review • Examination blueprinting (handout) • Accurate descriptions/interpretations (ET) • Internal structure (reliability e.g. GT)

  8. Validity testing by... • Response process – documentation of data quality control; rationale for using GRS vs checklist • Relationship to other variables (correlations) • Consequences – impact for examinees for assessment scores and outcomes (FP & FN) e.g. determination of the pass score, examiner qualifications, number of examiners

  9. Constructs • “intangible collections of abstract concepts and principles which are inferred from behaviour and explained by educational or psychological theory” • e.g. Educational achievement Educational ability (aptitude) Attitudes

  10. Domains in performance-based assessments Examples (handout) Clinical content – information gathering (Hx-taking) Clinical content – giving information Clinical content – communication skills Physical examination – clinical skills Procedure – clinical skills

  11. GRS • Checklist vs GRS • GRS – “subjective expert judgement” more reliable than checklists “use expert judgement for assessing complex skills” • Use of domains (handout with descriptors)

  12. Standard setting • Purpose - identify the cut-score, minimum competence required for progression/ graduation Competent Incompetent Pass Fail

  13. Standard Setting • Method has to be reliable, credible, supported by a body of evidence in the literature, feasible, acceptable to all stakeholders • Absolute standard/criterion referenced • Borderline regression method with GRS requires at least 4 domains of GRS + overall GRS e.g. Handout; at least 100 students

  14. Thoughts on the process • CONSENSUS • DOCUMENTATION • VEHICLE / STRUCTURE • INCREMENTAL • RESOURCES

More Related