1 / 49

Implementation of a Licensing Examination

Implementation of a Licensing Examination. Steven Lillis. Why assess?. Find those who are not yet competent Demonstrate to others that adequate systems are in place that promote public safety Reflection on appropriate standards for practice Feedback on teaching and curriculum. Why assess?.

napua
Download Presentation

Implementation of a Licensing Examination

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Implementation of a Licensing Examination Steven Lillis

  2. Why assess? Find those who are not yet competent Demonstrate to others that adequate systems are in place that promote public safety Reflection on appropriate standards for practice Feedback on teaching and curriculum

  3. Why assess? Find those who are not yet competent Demonstrate to others that adequate systems are in place that promote public safety Reflection on appropriate standards for practice Feedback on teaching and curriculum

  4. Purpose The purpose anchors the examination. Mission statement of a company “The mission statement should be a clear and succinct representation of the enterprise's purpose for existence”

  5. Purpose Examples: NZREX: “…candidates are competent to enter a period of provisional registration in New Zealand, during which time they will be further assessed.” RNZCGP: “…to be able to safely practice in advanced vocational education”

  6. Purpose PLAB: “Passing the PLAB test demonstrates that you have reached the minimum standard required to practise safely in the UK”

  7. Purpose (MCCQE)...assesses the competence of candidates who have obtained the medical degree, for entry into supervised clinical practice in postgraduate training programs, with respect to their knowledge, clinical skills and attitudes as outlined by the MCC objectives.

  8. Purpose Needs to reflect: Who is being examined (IMG with WHO recognised qualification) Why they are being examined (safety, competence) What standard they are being examined at (further supervision in a training scheme)

  9. Purpose

  10. Find those who are not yet competent What is the material to be tested? How are you going to test it? How do you know which candidates are acceptable?

  11. Find those who are not yet competent What is the material to be tested? How are you going to test it? How do you know which candidates are acceptable?

  12. What is the material to be tested? The crucial importance of blueprinting…… “An accurate image of a product”

  13. What is the material to be tested? The crucial importance of blueprinting…… “An accurate image of a product” “A sufficiently detailed plan that allows construction”

  14. Domains Cardiovascular Neurology Women's health Child health Respiratory Genitourinary etc etc etc Competencies Communication Problem Prioritisation Acute serious deterioration History Physical Examination Tests Procedures AXES

  15. The 3rd axis There will be a minimum of • 2 cases / 10% of the questions on elderly care • 3 cases / 17 % of the questions on women's health issues • 1 case / 5 % of the questions on preventative health issues

  16. Sample from blueprint... “The competencies that will be tested are: • Communication skills (2 cases) • Prioritisation of management (4 cases) • Recognition of acute serious deterioration and its management (2 cases), • History taking (4 cases) • Physical examination (3 cases) and • Interpretation of tests and procedures (1 case). There will be at least one case on child health, one case on mental health and one case on women’s health.”

  17. Sample from blueprint • All domains are covered at least once • All competencies are covered at least once

  18. Purpose Blueprint

  19. Find those who are not yet competent What is the material to be tested? How are you going to test it? How do you know which candidates are acceptable?

  20. Miller’s triangle

  21. Practicality for licensure examinations... • A written test…for material that is reasonable for this format • MCQ • EMQ • KFP • An OSCE…for material that is reasonable for this format

  22. Two different MCQs The first line treatment of staphylococcus skin infections is: • Penicillin • Co-trimoxazole • Flucloxacillin • Cefaclor • Nitrofurantoin

  23. Two different MCQs A 25 year old man presents 3 days after a puncture wound to his leg. Examination shows his temperature at 38.2 C and the appearance of his leg is:

  24. Two different MCQs Ass well as surgical drainage, the most appropriate antibiotic to prescribe is: • Penicillin • Co-trimoxazole • Flucloxacillin • Cefaclor • Nitrofurantoin

  25. Purpose Blueprint Implement

  26. Find those who are not yet competent What is the material to be tested? How are you going to test it? How do you know which candidates are acceptable?

  27. Which candidates are acceptable? The importance of the ‘cut’ score Implies the ability to rank candidates

  28. Which candidates are acceptable? Growing acceptance of combination of two methods

  29. Which candidates are acceptable? Growing acceptance of combination of two methods Angoff process: a group of experts who are familiar with the expected standard making a judgment about how a borderline candidate would score in each question Used in written or static OSCE stations

  30. Which candidates are acceptable? Growing acceptance of combination of two methods Contrasting groups: each candidate gets a score on a station as well as an assessment of “pass”…”borderline”…..”Fail” P B F Score

  31. Which candidates are acceptable? Growing acceptance of combination of two methods Contrasting groups: each candidate gets a score on a station as well as an assessment of “pass”…”borderline”…..”Fail” B Score

  32. Which candidates are acceptable? Growing acceptance of combination of two methods Contrasting groups: each candidate gets a score on a station as well as an assessment of “pass”…”borderline”…..”Fail” The mean of the borderline cases is taken…… Score

  33. Which candidates are acceptable? Growing acceptance of combination of two methods Contrasting groups: each candidate gets a score on a station as well as an assessment of “pass”…”borderline”…..”Fail” The mean of the borderline cases is taken…… …and readjusted for the error in the examination Score

  34. Why assess? Find those who are not yet competent Demonstrate to others that adequate systems are in place that promote public safety Reflection on appropriate standards for practice Feedback on teaching and curriculum

  35. External confidence Transparency and communication Release the blueprint Openness around expectations of candidates when in the examination Give pass rates Give methods of devising the ‘cut score’ Follow up of successful candidates Responsibility to failing candidates

  36. Why assess? Find those who are not yet competent Demonstrate to others that adequate systems are in place that promote public safety Reflection on appropriate standards for practice Feedback on teaching and curriculum

  37. Reflection on appropriate standards for practice Method of engaging many experienced clinicians in reflecting on appropriate standards for the profession “What counts as competence and by whom is something that the community negotiates over time; indeed, it is this negotiation that defines the community.” Eckert P, Wenger, E. What is the role of power in sociolinguistic variation? Journal of Sociolinguistics 9/4 2005 582-589

  38. Purpose Blueprint Implement Competence

  39. Why assess? Find those who are not yet competent Demonstrate to others that adequate systems are in place that promote public safety Reflection on appropriate standards for practice Feedback on teaching and curriculum

  40. Feedback on teaching and curriculum Assessment drives learning What information is given to potential candidates and why is this information given? Default curriculum May assist in courses aimed at exam preparation

  41. Purpose Blueprint Implement Safe medical practice Default Curriculum Competence

  42. Purpose Blueprint Implement Safe medical practice Default Curriculum Competence

  43. Purpose Blueprint Implement Safe medical practice Default Curriculum Competence An iterative process where the examination both tests and informs the profession about standards of competence

More Related