CMS Measure Development Education & Outreach

1. MACRA Info Session #4 CMS Measure Development Education & Outreach • Patient-Reported Outcomes in Performance Measurement Presenters: Brenna Rabel, MPH Nicole Brennan, DrPH February 13, 2019 2:00-3:00pm EST

2. Vision and Goals: MACRA Info Sessions CMS Measure Development Education & Outreach • An ongoing process to engage the public in quality measure development. • Elicit feedback that will help CMS design resources that can help all of those interested in healthcare quality improvement better understand the goals of quality measurement. • Education • Outreach • Dedicated Websites • Measure Development Roadmaps • Listserv opportunities

3. MACRA Info Session #4 CMS Measure Development Education & Outreach • Agenda: • Key definitions • Background • Benefits and challenges • Special considerations (conceptualization to testing) • Conclusion • References • Discussion

4. MACRA Info Session #4 Key Definitions

5. MACRA Info Session #4 Example • PRO: Patient’s experience of back pain • PROM: Patient’s assessment of their back pain collected via the Visual Analog Pain Scale • PRO-PM: Average change (pre-op to one year post-op) in back pain for adults who had a lumbar spine fusion surgery

6. MACRA Info Session #4 Background • PROMs are already common in clinical trials • Increasing interest in comparative effectiveness research, routine clinical practice, and EHR systems • Represent a key part of CMS’s Meaningful Measures Framework

7. MACRA Info Session #4 Benefits of PROs in Performance Measurement

8. MACRA Info Session #4 Challenges of PROs in Performance Measurement

9. MACRA Info Session #4 Considerations for Conceptualization

10. MACRA Info Session #4 Conceptualization of a PRO-PM (1/2) • Consider whether the outcome or condition of interest lends itself to patient-reporting • Information should be knowable by patients (ex., symptoms) • Outcomes and analysis results should be important and meaningful to patients • Measured outcomes should be sensitive to changes in clinical practice

11. MACRA Info Session #4 Conceptualization of a PRO-PM (2/2) • Conduct a thorough environmental scan • Identify PROMs that may serve as the basis for your performance measure • Consider whether they are validated, and for what population/clinical environment • Consider barriers to feasibility, such as patterns of use and how information from the PROMs are collected and stored • Determine whether to use a global or condition-specific tool • Compile a list of related or competing measures

12. MACRA Info Session #4 Final Considerations for Conceptualization • Rate the evidence supporting the selected tool • Determine whether the tool is in use and validated for your target population/setting • Consider whether the tool is suitable for use in performance measurement • Think through potential feasibility and implementation burden

13. MACRA Info Session #4 Considerations for Specification

14. MACRA Info Session #4 Specification (1/2) • Target population (e.g., patients 18 years of age or older who have undergone lumbar surgery) • Outcome type (e.g., average change, percent improvement, percent of patients who experience improvement) • Timing (e.g., within one month prior to surgery and between 270-365 days following surgery) • Risk-adjustment

15. MACRA Info Session #4 Specification (2/2) • Methods of administration (e.g., paper vs. electronic, in-office vs. at home via patient portal) • How proxy responses are handled • Response rate calculations • How responses affect results Specifications for PRO-PMs must also include:

16. MACRA Info Session #4 Considerations for Testing

17. MACRA Info Session #4 Evaluation/Testing • Same overall requirements as other outcome measures, with additional considerations • Importance • Feasibility • Scientific Acceptability (reliability and validity) • Usability and Use • Harmonization

18. MACRA Info Session #4 Importance Considerations • Must be patient-centered • Patients should be involved in selecting the outcomes and the PROMs for the performance measure • Must demonstrate that the measure can lead to improved outcomes and/or address a performance gap

19. MACRA Info Session #4 Feasibility Considerations • Burden to respondents • Questionnaire length • Timing • In-office assessment vs. online • Accessibility issues • Severity of patient’s condition • Type of illness

20. MACRA Info Session #4 Feasibility Considerations • Burden to clinicians • Workflow changes • Reliance on proprietary tools • IT requirements • Language, literacy, and cultural issues • PROMs available in multiple languages • Age of target population

21. MACRA Info Session #4 Scientific Acceptability Considerations (1/3) • Must be established for both the PROM and for the PRO-PM • Must demonstrate that the PROM tool is psychometrically sound • PRO-PM validity can be demonstrated by testing relationships (e.g., correlation) with another performance measure or PROM • PRO-PM reliability can be demonstrated using signal-to-noise (or similar)

22. MACRA Info Session #4 Scientific Acceptability Considerations (2/3) • Missing data may be more prevalent with PRO-PMs than with other types of performance measures • Missing responses on a multi-item scale • Missing responses from eligible patients • Missing information because of exclusions • Using proxies to mitigate potential missing responses • Analyses of missing data and response rates are required to demonstrate that potential problems in these areas do not bias the performance measure results

23. MACRA Info Session #4 Scientific Acceptability Considerations (3/3) • PRO-PM reliability and validity should be assessed quantitatively; requires data collection from test sites • Low sample size could jeopardize testing results • Carefully consider likely response rates when selecting sites to participate in testing • Review findings from feasibility testing to inform testing plans

24. MACRA Info Session #4 Usability Considerations • Demonstrate that clinicians can use the measure to improve the quality of the care they provide • Demonstrate that patients can use the information from the measure and find it to be meaningful

25. MACRA Info Session #4 Conclusions

26. MACRA Info Session #4 References • CMS MMS Blueprint • https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/MMS/MMS-Blueprint.html • National Quality Forum—Patient-Reported Outcomes • https://www.qualityforum.org/Publications/2012/12/Patient-Reported_Outcomes_in_Performance_Measurement.aspx • PCPI “Methods for Developing Patient-Reported Outcome-Based Performance Measures (PRO-PMs)” • https://doi.org/10.1016/j.jval.2015.02.018

27. MACRA Info Session #4 Discussion Questions

28. MACRA Info Session #4 Upcoming Info Sessions Planned Upcoming Webinars: • March 6, 2019 – Introducing the Clinical Quality Bundle and Pre-Rulemaking Updates • April 3, 2019 – Why Measures Fail Endorsement, presented by NQF To suggest topics for upcoming Info Sessions, email: MMSsupport@battelle.org

29. MACRA Info Session #4 CMS Measure Development Education & Outreach Contact information: • Battelle Measures Management System Contract Holder Contact: MMSsupport@Battelle.org • CMS Kimberly Rawlings: Kimberly.Rawlings@cms.hhs.gov