AcademyHealth, June 9, 2008 Washington, DC Weifeng Weng, Gerald K. Arnold,

1. Comparison of Physician Rankings on Performance Quality Composites in the Care of Hypertensive Patients AcademyHealth, June 9, 2008 Washington, DC Weifeng Weng, Gerald K. Arnold, Eric S. Holmboe, Rebecca S. Lipner

2. ABIM and Maintenance of Certification (MOC) • ABIM certifies physicians in internal medicine and its subspecialties • Certification is time-limited: 10 year duration • Renew certificates through MOC program • MOC requires demonstration of • Professionalism • Lifelong Learning • Cognitive Expertise • Practice performance

3. Practice Improvement Module (PIMTM) Evidence-based guidelines Based on Picker patient and CAHPS surveys Based on Wagner’s Chronic Care Model & IHI’s Idealized Office Design Patient survey Practice survey Chart audit Performance Report Improvement plan do act Impact study

4. Research Question • In P4P programs • Clinical measures dominate • Patient survey and Practice system survey measures used less frequently • Typically, rewards awarded by relative ranking • Typically not all three data streams used • Do physician performance rankings (and rewards) vary considerably when different combinations of the three data streams are used?

5. Methods • Physician database with patient-level data • Standardized composite scores for the three data streams (1) Chart audit (2) Patient survey (3) Practice systems • Super-composite scores: Combine composites • Examine changes in physician rankings

6. Physician and Patient Samples • 659 Physicians • Mean Age: 44 (SD = 6.4), 26% female • 61% general internists, 39% subspecialists (largely nephrologists and cardiologists) • 29% in solo practice • Patients • Chart audit: 13,096 patients, age 18-75, 51% male • Patient survey: 14,913 patients, age 18-75, 53% male

7. Chart Audit Individual Measures Outcome variables are risk adjusted for co-morbidity conditions: BP control <130/80 for pts with dm or stroke co-morbidities, <140/90 for rest. LDL control <100 for pts with major risks, <130 for pts with other risks, <160 for the rest.

8. Patient Survey Individual Measures

9. Practice System Individual Measures Practice system survey of 89 questions

10. Distribution of Physician Performance Composite Scores 6 4 2 0 -2 -4 -6 -8 C+S Chart (C) Patient (P) System (S) C+P C+P+S

11. Correlations among Composites

12. Percent who change rankings by more than one quartile* Baseline: Chart Ranks better than chart Ranks worse than chart * One quartile counts for 164 rank positions

13. Percent who change rankings by more than two quartiles* Baseline: Chart Ranks better than chart Ranks worse than chart * Two quartile counts for 329 rank positions

14. Examples of Extreme Discordance of Performance – more than three quartilesRanks and (z scores) Rank 1 = Best; Rank 659 = Worst

15. Conclusions • Measuring multiple dimensions in the quality of patient care is complex • Very moderate correlations among three data streams • Rankings change considerably depending on combinations • A profile that incorporates more than one aspect of patient care tells a different story than any one of them alone

16. Limitations and Future Research • Self-report data for chart and system data • Participants are volunteers • Need more robust risk adjusters • Investigate other analytic approaches for combining individual measures into composites • Investigate stability of pass/fail decisions