The National Board of Public Health Examiners (NBPHE)

1. The National Board of Public Health Examiners (NBPHE) Report to the ASPH Associate Deans Jack Barnette, PhD June 28, 2007

2. Status Report and Discussion • Want to provide a status report • Present issues and practices that currently or are likely to impact curricula and practice experiences

3. Purpose of the exam To ensure that students and graduates from schools and programs of public health accredited by the Council on Education for Public Health (CEPH) have mastered the knowledge and skills relevant to contemporary public health practice.

4. What will the exam cover? Five discipline-specific domains • Biostatistics • Environmental Health Sciences • Epidemiology • Health Policy and Management • Social and Behavioral Sciences Seven cross-cutting/interdisciplinary domains • Communication and Informatics • Diversity and Culture • Leadership • Professionalism • Program Planning • Public Health Biology • Systems Thinking

5. Past Efforts • Both ASPH and APHA established task forces on the credentialing of public health workers in the late 1980’s stimulated, in part, by the call from the U. S. Surgeon General for such an effort • In 1999, APHA and ASPH formed a Task Force on Public Health Workforce Credentialing

6. Recent Efforts • January 2002, ASPH’s Executive Committee approved moving forward to develop an independent Board of Public Health to issue examinations and provide those that pass the exam with a public health credential • National Board of Public Health Examiners incorporated in September 2005 • December 2005, inaugural meeting of the board held in Philadelphia, PA

7. Collaborating Organizations According to the by-laws, board members include representatives from: • APHA (2) • ASTHO (1) • NACCHO (1) • APTR (2) • ASPH (6*) • Up to 12 at-large members * Two of the ASPH-nominated individuals must represent private sector employers

8. Jack Barnette, PhD, MA Sr. Assoc Dean for Academic Affairs, UAB SPH Patricia A. Buffler, PhD Professor, Dean Emerita, UC Berkeley SPH Terry Dwelle, MD State Health Officer, ND DOH Mindy Thompson Fullilove, MD Columbia University Mailman SPH Kristine Gebbie, DrPH, RN Columbia University Mailman SPH Bernard D. Goldstein, MD (Chair) Professor, Dean Emeritus, Pittsburgh GSPH David I. Gregorio, PhD, MS Director, Grad Prog in Public Health, U CT Fernando A. Guerra MD, MPH Director, San Antonio Metropolitan Health District Sally Guttmacher, PhD Director, MPH Program, New York University Robert G. Harmon, MD, MPH Health Officer, Duval County Health Dept., FL Cheryl Healton, DrPH President and CEO, American Legacy Foundation Camara P. Jones, MD, MPH, PhD Research Dir, Social Determinants of Health, CDC Gerald T. Keusch, MD Director, Global Health, Boston University Edward M. Mamary, DrPH Director, MPH Program, San Jose State University Diane L. Matuszak, MD, MPH Director, Comm Health Administration, MD DHMH Alan Melnick, MD, MPH Health Officer, Clark County Health Department, WA Donna J. Petersen, ScD, MHS Dean, U. South Florida CoPH Andrew Sorensen, PhD, MPH President, University of South Carolina Adewale Troutman, MD, MPH, MA Director, Louisville Metro Health Department Walter Tsou, MD, MPH Past President, APHA, Public Health Consultant Board of Directors

9. Governance • NBPHE is an independent organization with a volunteer Board • Staff: • Interim President Dr. Charles Mahan, a nationally recognized leader in academic-practice linkages • Interim Administrator Cynthia Godes • Supported by ASPH senior staff

10. Standing Committees • Executive Committee: The Executive Committee is responsible for implementing the policies established by the National Board and has full authority to act for the National Board between the Annual Meetings. • Finance Committee: The Finance Committee shall review the annual budget of the NBPHE and recommend its adoption to the Executive Committee. The Committee shall advise the Executive Committee and the National Board concerning the financial status of the NBPHE and, upon request, advise the President concerning financial matters.

11. Standing Committees • Nominations Committee: The Nominations Committee is responsible for soliciting input and presenting slates of candidates for board members and elected officer positions, as well as honorary board members. • Testing Committee: The Testing Committee is responsible for oversight of all issues regarding exam development and delivery. • Test Materials Development Com. (Donna Petersen, chair)

12. Ad hoc Committees • By-Laws Committee: The By-Laws Committee is responsible for maintaining and updating the NBPHE by-laws. • Development Committee: The Development Committee is responsible for developing a plan to garner funding for NBPHE's operations. • Diversity Committee: The Diversity Committee will review all aspects of the NBPHE processes and exams to assure that issues of diversity (broadly defined) are appropriately considered and addressed.

13. Ad hoc Committees • Eligibility Committee: The Eligibility Committee is responsible for reviewing and making recommendations to the Board regarding criteria that individuals need to fulfill in order to sit for the credentialing exam. • Outreach Committee: The goal of the Outreach Committee is to develop plans for sharing information with all potential exam candidates and all who utilize and support public health expertise. • Research and Evaluation Committee: The Research and Evaluation committee will propose or review any NBPHE research and evaluation activities, and provide oversight for any needed data collection.

14. Eligibility for the Exam • Eligible examinees are masters and doctoral graduates of CEPH-accredited schools and programs of public health • A list is maintained on the CEPH website at http://www.ceph.org

15. Locations of CEPH-Accredited Schools and Programs

16. Timeline to the First Exam

17. Exam Development • NBPHE has contracted with the National Board of Medical Examiners to develop the exam, including: • A workshop to train test-item writers (held September, 2006) • Editing of items to assure high quality • Meeting scheduled for Sept. 2007 for item writers to review and revise items and develop first draft of exam • Various approaches to develop metrics that permit a valid and fair examination • Exam development based on sound principles of content validation

18. Competency Based ASPH competencies are being used as the framework However, item writers have not been asked to deal with specific competencies, but instead the broad core discipline and cross-cutting area Not easy to develop items that deal with both core and cross-cutting areas at the same time (ie. Biostatistics and biologic basis)

19. “Teaching to the Test” If the competencies are valid and we are testing the attainment of competencies, how can “teaching to the test” be a bad thing It would be much worse to not “teach to the test” if it is assessing important competency development It does tend to elevate the ASPH competencies in curricula

20. Curricula not based on the ASPH Competencies? • Personally, I view a great deal of overlap among the various competency sets • I don’t believe an examinee who has a public health degree from an accredited school or program that uses another competency set will be at any significant disadvantage • Having said that, I’d still feel a little more comfortable if my graduates were trained with a curriculum that was ASPH competency-based

21. Testing Preparation Activities Lot’s of interest in getting into the test-prep business One non-accredited school made the claim in its literature that it had designed curricula based on the exam Of course, this was false advertising and not based on fact since the exam was far from being developed Contact from our attorney took care of this problem Their graduates wouldn’t have been eligible to take the exam anyway

22. Testing Preparation Activities • I think some of the ASPH Councils are working on development of preparation materials • Perhaps Allison could fill us in on those activities

23. The Exam • Multiple-choice questions, some following cases or vignettes • Approximately 4½ hours to complete • Computer-based exam to be offered at testing centers in hundreds of locations nationally • Cost in the range of $300-400

24. Item Writers • Selection process • Solicited nominations from representative organizations • Received over 400 nominations • Characteristics of the 28 item writers • Gender: Female (15), Male (13) • Race/Ethnicity: Asian (1); Indian (1); Hispanic (2); Black (3), White (21) • Practice Experience (14) • Geographically diverse: Region 1 (2); 2 (2); 3 (3); 4 (10); 5 (1); 6 (3); 7 (3); 8 (0); 9 (2); 10 (2)

25. Standard Setting • I would have preferred that we give the exam to a group of graduates that could be used as a standard setting group • I would have preferred that we be able to examine psychometric properties of the items (difficulty and discrimination) to select items for the final exam • This has been deemed unnecessary by the NBME

26. Standard Setting • Also, this would be very expensive and was not accounted for in our budgeting • We will conduct a post-hoc item analysis and problem items will be deleted from the final scoring

27. Standard Setting • A method referred to as a modified Angoff method will be used where judgments of content experts relative to what proportion of examinees would be expected to get an item right will be used for standard setting • This group has not been identified as yet • I’m sure content experts from schools and programs will be sought to do this

28. Liaisons • Each CEPH-accredited school and program was asked to designate a liaison to NBPHE. • NBPHE staff will provide the liaisons with the most up-to-date information about the NBPHE credentialing exam, and have the opportunity for input into the exam development process. • The Liaison will be asked to keep the board informed of any concerns or questions from your faculty and staff. • Of course we welcome participation from all members of CEPH-accredited academic institutions. The liaison program is just one step in a series of mechanisms to involve the general public health community in this effort.

29. We will come visit your School • During this academic year, Board members are willing to come to your school to make presentations to students and faculty about at no cost to you • Just have your liaison contact us • We are also preparing informational materials for dissemination through student groups such as the APHA Student Assembly and at the APHA meetings (a booth and presentations scheduled for this fall)

30. Issues Under Consideration • Are the standards for public health practice being raised, and the professional development of public health practitioners being improved? • Is the test relevant to current public health practice? If it is not relevant, is the problem with the exam or what is being taught? • How will the exam serve as a tool for continuing education? What will recertification entail? • Will employers of public health graduates value the credential? • How will the exam be graded?

31. Conducting Research on the Exam • This will be a continuous activity of the Research and Evaluation Committee that I chair • We will look at: • How the exam operates with the examinees • Effects of the use of the exam on curricular changes • Effects of the use of the exam on how graduates are treated relative to employment, promotion, and income

32. Conducting Research on the Exam • Among the issues we will look at are the following types of issues: • Did there appear to be differential pass rates for different examinee subgroups (we are particularly interested in any evidence of cultural/ethnic bias) • Are the score differences between graduates from different discipline areas • Are there score differences between accredited schools or programs

33. Conducting Research on the Exam • We will be very cautious and sensitive about dissemination of results • Any dissemination will be reviewed and approved by the NBPHE before release

34. How Curricula will be Impacted • Indirectly, just the plan to develop and offer this exam is having an impact on curricula • The consistency of the adoption of the ASPH competencies in curricula and the use of them as a framework for the exam impacts curriculum • As the exam becomes more a part of the practice of credentialing, the impact is likely to be greater

35. How Curricula will be Impacted • However, keep in mind our intent is not to directly affect curricula, but do document that graduates of our schools and programs have a basic set of competencies to be able to practice public health • If curricular changes are needed to improve the likelihood that this is happening, the NBPHE is all for it

36. How Practice Experiences will be Impacted Trying to include items that will require some problem solving and interpretation of the real world of public health practice Items higher in Bloom’s Taxonomy than recall These types of items can influence practice experiences May be a few years off before we see much of this as we get better and assessing these types of competencies

37. Reporting Results Each Examinee will be provided results for each core area and a profile of passing the cross-cutting areas Not decided whether will report out by % correct or by a percentile rank Not decided how we will report to Schools and Programs

38. Grandfathering • The NBPHE has decided it will not grandfather members of the current public health workforce • This would be a very difficult and costly process that we cannot accomplish with our resources

39. Use of Exam as a Program Exit Exam • Some have thought the exam could be used to assess program competency attainment at the end of the program as an exit exam • This will not work since only graduates can sit for the exam

40. What about Recertification? • The NBPHE is currently looking into this issue • It is likely we will promote a continuing education type program to do this • Schools and programs are likely to have the opportunity to provide such opportunities to individuals who seen recertification

41. What about a Spanish version? At the most recent NPHBE meeting, held just a few weeks ago, this issue was brought up I would like to discuss this with Laura and Jose at this meeting Bernie Goldstein will be talking with the Puerto Rico and Cuernavaca deans about doing this We are looking into the costs and logistics of doing this I’m not sure we will have the exam given in August 2008 in Spanish, but hopefully we will offer that in 2009 It needs to be done

42. Contact Us The National Board of Public Health Examiners 1101 15th Street, NW, Suite 910 Washington, DC 20005 voice: (202) 296-1099, ext. 132 fax: (202) 296-1252 e-mail: info@nbphe.org Website: http://www.nbphe.org/