1 / 18

Nephrology Knowledge Based Assessment (Specialist Examination, MRCPUK(Nephrology))

Nephrology Knowledge Based Assessment (Specialist Examination, MRCPUK(Nephrology)). Jonathan Fox March 2008. Specialist Examinations: purpose. To ensure that certified specialists have sufficient knowledge to practise competently and safely as consultants

mkruse
Download Presentation

Nephrology Knowledge Based Assessment (Specialist Examination, MRCPUK(Nephrology))

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Nephrology Knowledge Based Assessment(Specialist Examination, MRCPUK(Nephrology)) Jonathan Fox March 2008

  2. Specialist Examinations: purpose • To ensure that certified specialists have sufficient knowledge to practise competently and safely as consultants • To complement workplace-based assessments • To work with specialist societies and SACs provide a rigorous national assessment to establish public confidence • To offer a challenge similar to subspeciality certification exams in North America

  3. Specialist Examinations: delivery • 1 diet per year (was to be 2 per year) • Computer-based testing (www.pearsonview.com) • 2 papers in each diet • 100 best-of-five questions in each paper • Assess core knowledge and application of this knowledge in a clinical setting • Strict distribution of questions in accordance with blueprint derived from specialty curriculum • To be taken in ST4 usually and required for CCT

  4. First wave Dermatology Gastroenterology Geriatric Medicine Neurology First diet June 2008 Second wave Acute medicine Cardiology CPT Endocrinology/diabetes ID Medical oncology Renal medicine Respiratory medicine Rheumatology First diet November 2008 Specialties

  5. Timeline: medical specialties • 2004-6: JCHMT Pilot (report Oct 2006) • late 2006: MRCP(UK)/Federation of Medical Royal Colleges proposed partnership with specialist societies • early 2007: Appointment of Examination Board Chairs & Secretaries • June 2008: First wave examinations • November 2008: Second wave examinations

  6. Timeline: Nephrology • March 2007 Appointment of Chair (J Fox) & Secretary (J Levy) • March 2007 advertisement for QWG members (41 responses, 5 not requiring training) • 18 July 2007: first training day – 24 attended • 5 Sept 2007: second training day – 11 attended • 8-9 Jan 2008: Question Writing Group meeting – 26 attended, approx. 333 questions produced • 3-4 June 2008: Board meeting • Aug/Sept 2008: Standard Setting Group meeting • November 2008: KBA

  7. Question Writing Groups • 2-day meetings twice a year initially • Consultants • Wide geographical spread • Some from MRCP SQG • Attended workshop • 30 questions per year per member • Guidance from Secretary/Chair on topics • Stand down if fail to fulfil commitment

  8. A 34-year-old woman was referred for the investigation of bloody diarrhoea. She was opening her bowels four times daily. On examination, she was well. Her pulse was 64 beats per minute and her abdomen was soft and non-tender. Colonoscopy revealed an active colitis, limited to the sigmoid colon. Biopsies confirmed a diagnosis of ulcerative colitis. Investigations: haemoglobin 132 g/L (115 - 165)white cell count 11.5 x 109/L (4 - 11)platelet count 323 x 109/L (150 - 400) serum albumin 40 g/L (37 - 49)serum C-reactive protein 13 mg/L (<10) What is the most appropriate initial treatment? A mesalazine enemasB oral azathioprine C oral mesalazineD oral modified-release budesonide E oral prednisolone Answer Key: A

  9. The Challenge of Question-Writing • 200 questions per year • Curriculum coverage • Each question should not be re-used more often than once every 3 years • Question bank should contain at least 1000 usable questions To generate one re-usable question for MRCP(UK) requires 3-5 questions to be drafted

  10. Question production process QWG members Non-medical editors QWG secretaries QWG meeting Final vetting by chairman/secretary Question Bank Examination Board Standard Setting Group Examination

  11. Board Roles: • To set papers for each examination • To oversee delivery of examinations • To be responsible for academic matters, misconduct, complaints, regulatory matters Composition: • 10 members incl. chairman & secretary 2 represent SAC 4 from the Question Writing Group 4 non-writing members Meets for 2 days a year

  12. Standard Setting Group Role: • To set pass mark for the exam (modified Angoff method, Hofstee compromise applied after exam) • To develop assessment strategy Membership: • 6 members incl. Board Chair & Secretary • No-one else should belong to both QWG & Board • Chair should have experience of standard setting Meets for 2 days a year

  13. Why collaborate with MRCP(UK)? • Common approach for 13 medical specialties • Format used since 2002: Part 1: 3 diets of 2 papers (200 Qs) per year Part 2: 3 diets of 3 papers (~270 Qs) per year • Medical experience: SQG, Board, Standard Setting • Statistical & psychometric support • Administration: non-medical editors, organisation of meetings, etc • IT: question bank & CBT

  14. Reliability (Cronbach’s alpha): MRCP(UK) Part 2 Written Examination

  15. Pass rate Proposed: >85% per diet ~98% overall Desired pass mark?

  16. Challenges • Heavy workload for a relatively small specialty • Small number of candidates (cf 1245 candidates for MRCP Part 2, 2007/2) will make pass mark setting, reliability assessment and pre-testing of questions difficult • Name (MRCPUK(Nephrology)) • Cost to candidates/affordability to RA

  17. Links • http://www.jrcptb.org.uk/assessment/Pages/MRCP(UK)KnowledgeBasedAssessment.aspx • http://www.jrcptb.org.uk/SiteCollectionDocuments/KBA%20Project%20Final%20Report.pdf • www.pmetb.org.uk/fileadmin/user/QA/Assessment/Assessment_good_practice_v0207.pdf • http://www.mrcpuk.org/Pages/Home.aspx

More Related