Standardization of Pharmacist Competency: The 2009 National Pharmacy Practice Survey and NAPLEX Standard Setting

1. Standardization ofPharmacist Competency:The 2009 NationalPharmacy PracticeSurvey andNAPLEX Standard Setting NABP President, William T. Winsley, MS, RPh NABP Competency Assessment Senior Manager Maria Boyle, MS, RPh

2. North American Pharmacist Licensure Examination®(NAPLEX®) Purpose: To assess a candidate’s competence, knowledge, skills, and abilities to practice pharmacy

3. Licensing Examinations • Measure skills directly related to the purpose of licensing (protection of the public) • Content domain should be clearly defined and justified in terms of its importance in a given profession • A survey of practitioners provides the basis for defining the balance of content within an examination • Serves as one criteria to make decisions regarding professional practice • Professional test developers follow standardized practices to ensure that tests measure what they purport to measure

4. 2009NABP National PharmacyPractice Survey

5. Methodology • Suggested updates to the competency statements are reviewed by the NABP Advisory Committee on Examinations and the Executive Committee • Once the updated competency statements are approved, they are used to construct a practice analysis survey • In the 2009 update, recommended changes included the addition of pharmacoeconomics to three competency statements as it relates to clinical care and pharmaceutical outcomes • The survey is piloted among a subpopulation of pharmacists before being launched as a large-scale survey

6. Methodology • NAPLEX competency statements are continually evaluated by subject matter experts (SMEs) • All SMEs are practicing pharmacists and/or academicians • An advisory committee of SMEs oversees the process of the competency statements review and the practice analysis • SMEs review appropriate literature and provide perspectives as incumbents to gather information to support the content domain description

7. 2009 Sampling Population • NABP pharmacist database • National Association of Pharmacy Regulatory Authorities (NAPRA) • Pharmacy Practice faculty • APPE Preceptors • AACP assisted NABP: • by identifying a sample of pharmacy practice faculty to solicit as academic participants, and • by forwarding an invitation to participate to the Directors of experiential education at colleges of pharmacy to assist in targeting experiential preceptors • 82.6% of US respondents indicated “preceptor experience” in the last five years

8. Survey • Online survey was available from April to mid-July 2009 • Respondents were asked to rate each competency statement on two scales: Criticality and Frequency • A five-point rating scale was used: 1 (low) to 5 (high)

9. Survey Rating Scales

10. Examples of Competency Statements Identify and assess patient information including medication, laboratory, and disease state histories Identify specific uses and indications for drug products and recommend drugs of choice for specific diseases or medical conditions See the NAPLEX/MPJE registration bulletin at www.nabp.net

11. DemographicsofRespondents

12. Total Respondents 2,958 United States 2,244 Canada 714

13. Year of Licensure

14. Professional Degree 34% PharmD 48% BS/BPharm 16% Earned both professional degrees 2% “No response” 27% Reported completing residency

15. Primary Practice Setting

16. Ethnicity and Gender

17. Analysis • Survey data were analyzed to determine measures on items and to translate those measures into content weights for the test blueprint • Proportion of weights based on criticality vs frequency were made • The NAPLEX Review Committee and the Advisory Committee on Examinations reviewed the outcomes and made recommendations to the Executive Committee

18. NAPLEX Blueprint Changes • Area 1: Assess Pharmacotherapy to Assure Safe and Effective Therapeutic Outcomes • Increased from 54% to 56% of examination (equates to three additional items delivered) • Competency 1.2 now includes pharmacoeconomic factors as they relate to the identification and selection of pharmacotherapeutic agents • Competency 1.3 now includes the evaluation and management of drug regimens to enhance safe, effective, and economic patient outcomes

19. NAPLEX Blueprint Changes, cont. • Area 2: Assess Safe and Accurate Preparation and Dispensing of Medications Decreased from 35% to 33% of examination • Area 3: Assess, Recommend, and Provide Health Care Information that Promotes Public Health Competency 3.1.2 now includes the evaluation of pharmacoeconomic data as it applies to patient care

20. What’s New with the NAPLEX? • New competency statements and blueprint went into effect March 1, 2010 • A new passing standard was set • Variety of item types which include selected-response and constructed response test items • Enforcement of score invalidation in cases regarding candidate misconduct

21. NAPLEX Standard Setting

22. Establishing a Cut-Score • Critical test development issue, particularly in high-stakes, criterion-based testing • Legitimacy of the standard setting process is a significant component of validity evidence • What degree of mastery over the content domain (knowledge, skills and abilities) is needed to establish competency?

23. Overview • Panelists (NAPLEX Review Committee, practitioners, academicians, preceptors) were recruited for a two day workshop • NABP provided training on test development and standard setting • Practice examples, group discussions, and opportunities to make practice ratings were provided

24. Standard Setting Session • Panelists were asked to conceptualize the knowledge, skills, and abilities necessary for entry-level practice • Key considerations focused on public health and safety

25. NAPLEX Standard Setting • Rater training • Standard setting process • Expectations • How data will be used • Defining a qualified candidate • Characteristics of a sufficiently knowledgeable yet minimally competent candidate (SKYMC) were discussed • Item ratings • Judges consider, answer, and rate real NAPLEX items to make judgments on what a SKYMC candidate would get correct

26. NAPLEX Standard Setting • Preliminary statistical analysis of all of the data collected was performed • Feedback and impact data were reported to the panelists • Rating process was repeated and resulting data analyzed • Considerations for inter-rater reliability • Considerations for subjectivity regarding pass rates • Discussion of outcomes among judges

27. Policy and Recommendations • Ranges of potential cut-scores were computed; an impact analysis was conducted • The NAPLEX Review Committee and the Advisory Committee on Examinations analyzed outcomes and impact data • A recommendation was made to the Executive Committee for a final decision

28. Program Development

29. The Pharmacist Assessment forRemediation and Evaluation (PARE) • NABP Executive Committee recognized the need for an objective assessment to address situations of questionable pharmacist competency and remediation • The purpose of the assessment is to provide a multidimensional assessment for the boards of pharmacy that can be utilized as a contributory factor in making decisions about conditional pharmacist practice issues

30. The Pharmacist Assessment forRemediation and Evaluation (PARE) • Intended for experienced practitioners • Content domains: • Clinical pharmacy • Medication safety management • Professional ethics • The same industry standards applied to all NABP examination and assessment programs shall be implemented • Expectation for a 2011 launch

31. Questions?Thank you!