MDCB Examination: A History

1. MDCB Examination: A History Robert D. Adams Assistant Professor UNC School of Medicine Department of Radiation Oncology

2. Recognition • Damien Pusey • Lisette Smith • Dr. Elaine Zeman • MDCB Board of Directors: Past and Present

3. Why Give a Certification Board Exam? • Has grown exponentially last 100 years (Flexnor Report) • Is a very important issue: judge knowledge and abilities • Advances the standards and determines competence in health care delivery • Encompasses both level of education and work qualifications of a medical professional

4. Why Give a Certification Board Exam? • Helps shape the scope and practice of medical professionals and the care they provide • Influences the way health insurance sets standards for reimbursement practices

5. What is Certification? • “Program and process where a learner completes prescribed training and passes and assessment with a minimal score.” • Increases the validity of the profession

6. What is training? • Developing skills that will be used more by society than for the self (education) • Very important concept in higher education • Must be increased as we progress from an industrial society to a technological society

7. Training Vs. Education • Training deals with learning specific skills • Education focuses more on the development of the mind and broad-based thinking

8. Importance of Board Certified Medical Dosimetrists • Play an integral role in the treatment of cancer • Require high accuracy and precision to fulfill the job responsibilities • Important for credibility of profession

9. The Importance of Board Certification for Medical Dosimetrists • The role of a medical dosimetrist is important and, although a part of a team, is autonomous for billing purposes • Due to a high job responsibility, it is in our best interest that medical dosimetrists are board certified

10. Importance of Board Certification with Training • Utilizes specific skill sets • Impacts public health • Impacts the quality of health care delivery

11. Importance of Board Certification • Enhances the utility of society • Allows us to define ourselves, our skill levels, and our profession through a board certification examination

12. Why is the Dosimetry Exam Autonomous? • Why not under the AAPM? • Why not under ASTRO? • Why not under the ARRT?

13. Vision • Members of the AAMD had a different vision 25 years ago • Not to have the exam as being under someone else • The vision was to have an autonomous exam run by peers

14. Vision • In order to achieve the vision, there had to be great leadership • Members stepped up to create an autonomous exam, run by peers, with the help of professional company • The driving force to do this came from your professional society, the AAMD

15. MDCB • Incorporated in 1988 • Mandated to create and implement standards of certification in medical dosimetry

16. Goals of MDCB Goal 1: Elevate the standards and advance the cause of medical dosimetry by encouraging its study and improving its practice

17. Goal 2: Determine the competence of medical dosimetrists and to conduct examinations to test the qualifications of voluntary candidates

18. Goal 3: To grant and issue certifications in the field of medical dosimetry to eligible voluntary applicants and maintain a registry of holders of such certificates

19. Goal 4: To serve medical dosimetrists and the associated health care community by maintaining a Registry of Certified Medical Dosimetry

20. The Exam • First given in 1988 • Have given 21 exams • Given annually • Given in June because it coincides with the AAMD meeting • Written format

21. 2008 Exam Statistics • Physics 31 18.8 4.57 • Dose Calculation 39 24.4 5.46 • Treatment Planning 47 27.3 4.94 • Localization 12 8.1 1.85 • Brachytherapy 8 3.9 1.31 • Radiation Protection 3 1.9 0.77 • Quality Assurance 5 3.1 1.01 • Prof. Responsibility 3 2.6 0.40 • Computers / Network 7 3.2 1.15 Totals 155 93.3

22. Exam Lingo • R-Biserial scores (internal) • Subkoviak scores (twice) • Kuder Richardson Formula 20 scores (candidate differentiation) • Measuring the reliability and validity of the examb

23. The MDCB Board of Directors • Comprised of 12 members: • 6 CMDs • 2 Medical Physicists • 2 Radiation Oncologists • 1 Testing Person • 1 Community at Large Person

24. The MDCB Board • Each person volunteers to serve on the Board • Depending on the Board position and type of work, Board Members volunteer anywhere between 100 to 400 individual hours per year to better serve its members

25. The MDCB Board The MDCB Board employs TWO separate companies: 1) Association Headquarters • Management Company • Does renewals, handles your questions, gives out the certifications, tracks continuing education

26. The MDCB Board 2) Prometric • Testing Company • Also gives the MCAT, LSAT, and is a part of ETS • The MDCB works with this company to give a professional examination • As testing chair, this is who I primarily work with throughout the year

27. The Exam: Facts • 155 Questions (potential for increase) • All questions are written or peer-reviewed by the MDCB Board of Directors • Item bank of over 1,000 questions • Each Board Member required to write 20 questions each year • MDCB looking for question writers

28. The Exam: Review • The MDCB Board of Directors meets 3 times per year: 1) January: review new questions 2) March: review upcoming examination 3) August: review June exam results and prospectively begin for the next year’s exam

29. The Exam: Content • Content areas • 9 areas • The most recent area of inclusion is computer questions (2005) • Content areas are based upon surveys sent to working medical dosimetrists and their job tasks

30. The Exam: Content (cont.) • Based only on the medical dosimetry work surveys and the percentages of work • Updated every 5 years • Currently being completed is a job task analysis (work survey) will be implemented for the 2010 exam

31. The Exam: Qualifications • To take the MDCB Certification Exam you must: • Graduate from an accredited education program AND have 6 months clinical experience (up for discussion) • Have 24 months OJT and 12 continuing education hours coupled with a baccalaureate degree in the physical sciences or a radiation therapy certification (entry level examination) (possibly lengthen) • Route 3: possibly delete this route

32. The Exam: Pass Rates • Most controversial component of the Exam • Current pass rates are around 57% (past 5 years) • Pass rates are higher for examinees who attend JRCERT accredited educational programs versus on the job training

33. The Exam: Difficulty • The Exam is difficult • It fits in the ‘middle’ of certification exams nationwide. For example: • The CPA exam has a 27% pass rate • The ARRT Radiation Therapy Exam has a 90% pass rate

34. Why is the exam so difficult? • Not an entry level examination; it is written and there is no oral component to the exam • Assumes a high level of clinical competence coupled with high didactic abilities • For example, the ARRT Radiation Therapy Examination is an entry level examination, thus the higher percentage pass rate

35. Why is the exam so difficult? (cont.) • The MDCB Board makes no excuses about the complexity of this examination • The examination is not designed to be an entry level examination • The examination is designed for skilled, trained, and educated (theory) medical dosimetrists • The bar is set high and it will remain high

36. The Exam: Goals for the Future • A major goal of the MDCB Board of Directors is to take the exam from a Written format (pencil and paper) to a Computer Based Testing format • This goal is close to being achieved

37. The Exam: Goals (cont.) • Pilot testing will begin this year with the MDCB Board Members taking the computer based exam • Pilot testing will continue through 2010 • The goal is for the first comprehensive computer based exam to be given in 2011 during the June administration

38. The Exam: Goals (cont.) • Achieving this goal has required lots of volunteer individual work from both present and past board members

39. Computer Based Testing

40. Testing Models I. Linear CBT • Predetermined, linear order • Sequentially administered • Like our current paper exam given on a computer • Weakness: security concerns in that everyone has the same questions

41. Testing Models II. Linear on the Fly Testing (LOFT) • Build models based on content and psychometric targets • The items are scrambled • For example, Candidates A and B will not have the same questions

42. Testing Models III. Computer Adaptive Testing (CAT) • Adaptive to the candidates performance on the exam • Post-tested items are placed on the exam • Exam can be reduced by 50% • Increases security dramatically

43. Scoring Models I. Classical Test Theory (CTT) • This is what we currently use • Single performance score +% error • P values for item difficulty • Score correlations: R biserial • Simple to do and understand

44. Scoring Models II. Item Response Theory Model (IRT) • One Parameter and Three Parameter models • One Parameter is based on the items and a total score • Three parameter is based on item responses and not the total score: two candidates with the same score; one might pass and one might not if one candidate only answers simple versus medium or hard questions

45. CBT • Goals: • Examinees will be able to test at over 100 sites 2) The time from taking the exam to receiving a score within 5 years will be automatic (versus 10 to 8 to 7 to 6 weeks) 3) You may not have to take the entire exam; it will cut off when you have passed

46. CBT 4) Exam will become more secure • No paper • Eventually fewer questions • Different questions given to various test takers 5) Exam given multiple times throughout the year The CBT is going to literally take the exam to a different level of testing and scoring

47. Where we have been, where we are, where we are going; then? • Biggest Threat to Your Exam? • Exam Security

48. Security • External countries question writers • Therapy Exam story • Prometric and the FBI • Review courses: questions • Your responsibility • Your credential

49. Computer Based Testing

50. Final Thoughts • Storied history • Exam continues to be reliable and valid • Exam is moving in a forward direction • Security is the biggest deterrent to keeping the exam