Methodological Issues in Measuring Physician-Level Quality and Efficiency

1. Methodological Issues in Measuring Physician-Level Quality and Efficiency Ateev Mehrotra MD MPH RAND Health & University of Pittsburgh AcademyHealth Annual Research Meeting June 5th 2007

2. Applications of Physician Level Profiles • Public reporting – information to help people make more value-based decisions • Pay-for-performance – financial rewards to physicians with better performance • Tiering – differential co-payments tied to physician performance • $10 to see an “above average” efficiency & quality physician • $30 to see a “below average” efficiency & quality physician

3. Overall Project Goals • Identify key methodological choices that arise when constructing physician quality and efficiency profiles • Evaluate whether decision on methodological choice results in physicians being placed in different categories • Identify potential policy impact when applying these metrics

4. Methodological Issues on Efficiency Profiles Being Addressed • Constructing efficiency scores • Categorizing physicians into categories • Evaluating effects of reimbursement versus utilization on efficiency scores • Examining the relationship between efficiency and quality • Evaluating different units of analysis • Assessing alternate attribution rules

5. Overall Findings • Number of choices necessary when creating these profiles • Approaches used are rarely transparent to users or those being evaluated • Empirical evidence that choice matters

6. Data Source • 2003 & 2004 claims from 4 major health plans in Massachusetts • 2.9 million commercial enrollees • Adults <65 who were continuously enrolled for two years • Aggregated database ~90% state’s commercial health plan market

7. Quality Measurement Approach

8. RAND QA Tools • Subset of the measures used in RAND’s national study of health care quality • Claims-based algorithms • 129 measures of technical process quality across 23 conditions

9. Efficiency Measurement Approach

10. Symmetry’s ETG • Commonly used program among health plans • Patient’s claims aggregated into episodes of care • Episode of care is all care provided over a period of time for a specific condition • e.g. Pneumonia – first through last claim for pneumonia-related care • e.g. Diabetes - all diabetes care received in year

11. Physician’s Efficiency Profile • Each episode assigned to the physician using an attribution rule • Calculated for each episode: costs of given episode (observed) ----------------------------------------- average costs for that type of episode across all patients (expected) • Overall score for a physician is the ratio of observed to expected costs across all assigned episodes

12. Analytic Sample

13. Focus on Attribution

14. What is Attribution? • How do we decide which MD is responsible for care? • Except when there is a contractual relationship (gatekeeper), most approaches are algorithmic • Explore different algorithms and empirical impact

15. Choice #1 Level of Analysis? Patient-based MD is responsible for managing overall care for patient Episode-based MD is responsible for managing a condition or problem

16. Choice #2 What is Signal for Responsibility? Triggering Event Visit that started the episode (vs. who played the most) E&M Visits Evaluation Costs Professional services & Rx Majority (>50%) Majority (>50%) Plurality (>30%) Plurality (>30%)

17. Choice #3 One or Multiple Physicians? Single MD One MD is responsible for managing patient or condition Multiple MD Team approach to managing a patient or condition

18. Focus on One of these Choices:Level of Analysis Patient-based Episode-based

19. Number of Doctors Seen in 2003-2004 for E&M Visits EPISODE-BASED PATIENT-BASED

20. How Often Do Different Attribution Rules Assign the Same Episode to the Same Physician?

21. Half of Massachusetts MDs Classified Differently Under 2 Rules Patient-Based Episode-Based

22. Sample SizeAverage # of Total Episodes Assigned to Different Specialties urology, neurosurgery, plastic, vascular, thoracic

23. What Percent of Episodes Relevant to Specialty

24. Policy Implications • No “right” approach for attribution as it depends on policy goal and desired behavior change • For tiering, patient-based might be best • Patient usually chooses a primary provider • Primary provider has a set referral network • For P4P, episode-based might be best • Locus of control • Shared responsibility

25. Study Team • Elizabeth A. McGlynn, Ph.D. • Ateev Mehrotra, M.D. • Bill Thomas, Ph.D. • John Adams, Ph.D. • Scott Ashwood • Rodger Madison • Julie Lai • Fuan-Yue Kung mehrotra@rand.org For More Information

26. For more information

27. How Often Does only Single MD Care for a Given Episode?

28. Most Plans Contract with Physicians Statewide

29. But Utilization Is Concentrated in One Area of the State

30. So Aggregating These Plans May Not Increase Sample Sizes

31. Or Any Two of These Plans

32. But Aggregating Data with this Purchaser Increases Number of Observations

33. Average # of Episodes Assigned to Different Specialties Urology, Neurosurgery, plastic, vascular, thoracic Ob/Gyn, Cardiology, Neurology

34. Relationship Between Quality & Cost-Efficiency High Quality Score Low Efficient Inefficient Cost-efficiency Score

36. For Many Physicians, We Do Not Have Enough Information to Create Robust Profiles

37. Scores Are Based on a Minority of Patients and Claims Claims Enrollees All Enrollees 100% 100% Limited to Adults 18-65 88% 67% Limited further to those continuously enrolled 46% 39% Limited further to those with at least one claim 32% 46%