Patrick S. Romano, MD MPH Julie A. Rainwater, PhD Jorge A. Garcia, MD MS Debora A. Paterniti, PhD

1. A Randomized Controlled Trial of an Educational and Motivational Intervention to Enhance Consumers’ Use of Health Plan and Medical Group Quality Data Patrick S. Romano, MD MPH Julie A. Rainwater, PhD Jorge A. Garcia, MD MS Debora A. Paterniti, PhD Daniel J. Tancredi, MS PhD Geeta Mahendra, MS Jason A. Talavera, MD student AcademyHealth 2006 ARM June 27, 2006 Seattle, WA

2. Background to INQUIRE (INformation about QUality In a Randomized Evaluation) • Most previous studies of how consumers use report cards were conducted in “laboratory” settings, relied entirely on survey data, or did not randomly allocate participants. • We planned a prospective study with 3 components: • Focus group discussions of consumer choice and quality of care • A prospective cohort study of factors associated with using a quality report card and switching health plans/medical groups • A randomized controlled trial of two interventions designed to improve the use of quality information, under the Health Belief Model. • Funding from the US Agency for Healthcare Research and Quality

3. Health Belief Model

4. Phase II – Overview of Methods • Partnership between UC Davis and Pacific Health Advantage • Population: 76,000 employees of small businesses (with 2-100 eligible workers) in CA, excluding “guaranteed associations” • Setting: Open Enrollment 2003; members were offered a choice of 4 statewide and 4 regional HMOs (each with 3 copayment levels), 1 PPO (with 3 copayment levels), 1 point-of-service plan • Study design: Randomized controlled trial • Unit of randomization: Health insurance brokers (with their contracted employers and their employees) • Measures: Observed behavior, post-Open Enrollment survey

5. Phase II – Control group • Control group received “usual care”: • Open Enrollment booklets on program rules and benefit options were mailed to employers (no quality information) • No information went directly to employees • PacPlan Chooser web site allowed members to compare plans on cost, features, and quality (overall rating) • Insurance brokers provided limited support

6. Phase II – Interventions • Intervention group 1 received “educational/motivational treatment”: • A special mailing to each employee, employer, and broker included a motivational letter (with negative framing), the California HMO Report Card, and the California HMO Guide • A toll-free telephone line and e-mail address were offered for counseling and advice (during business hours) • Intervention group 2 was delayed

12. Phase II – Sample design • Stratified random sample of brokers with eligible employees scheduled for Open Enrollment in May-July 2003, after excluding employers intending to leave (N=1,579 with 26,249 EE’s) • Excluded 16 brokers with large number of eligible employees (to increase efficiency) • Oversampled small brokers (4 strata), brokers for whom at least 40% of EE’s were <39 yrs, and brokers for whom at least 50% of EE’s had 3 or more HMO options (total 10 sampling strata) • Brokers allocated in two stages

13. Phase II – Analytic methods and hypotheses • All analyses were (or will be) weighted to account for the cluster sampling design, using robust methods to correct CIs • Hypotheses: • Intervention would increase overall switching across health plans and medical groups • Intervention would promote switching toward “better” health plans and medical groups, among those who switch • Intervention would enhance perceived threat, enhance self-efficacy, promote migration from pre-contemplation to contemplation, and promote use of quality information in decision-making

14. Phase II – Process results • 292 brokers with 1,835 eligible employees (EE’s) were randomized to the intervention group • 246 brokers with 1,578 eligible employees (EE’s) were randomized to the control group • 30.2% of EE’s in the intervention group, and 37.1% of EE’s in the control group, dropped out of Pacific Health Advantage • 22 intervention group members used the toll-free advice line • 3 intervention group members used the e-mail address • Broad array of questions and concerns

15. Phase II – Primary outcome results (all weighted and nonsignificant) • 9.2% of intervention group versus 7.0% of control group switched plans. • 21% of intervention group switchers versus 35% of control group switchers moved to a plan with more stars. • 27-28% in both groups moved to a plan with fewer stars.

16. Secondary outcome, use of resources

17. Secondary outcome, reason for switch Stated reason for switching in intervention group vs. control group, respectively (all p>0.10 unless stated): • Change in geographic coverage of plan (10% vs. 2%, p=0.03) • Cost (34% vs. 25%) • Continuity of MD (1% vs. 5%) • Better network of MDs (8% vs. 7%) • Concern over poor report card scores (6% vs. 1%, p=0.099) • Concern over poor access to care (6% vs. 5%) • Poor service from previous plan (6% vs. 3%) • Better benefits (5% vs. 3%) • Other reason (7% vs. 1%, p=0.06)

18. Secondary outcome, expected outcome of switch (NS)

19. Secondary outcome, considered switch • 35% of intervention group respondents who did not actually switch “considered” switching • 28% of control group respondents who did not actually switch “considered” switching (p=0.07) • Of those who “considered” switching, 31% of intervention group respondents and 30% of control group respondents “seriously considered” it (rating=6 on 1-6 scale)

20. Secondary outcome, reason for considering switch Stated reason for considering switching in intervention group vs. control group, respectively: • Cost (69% vs. 74%) • Continuity of MD (6% vs. 11%) • Better network of MDs (16% vs. 23%) • Concern over poor report card scores (15% vs. 7%, p=0.08) • Concern over poor access to care (17% vs. 17%) • Poor service from previous plan (10% vs. 10%) • Better benefits (25% vs. 25%) • Other reason (14% vs. 7%) – need to review comment fields

21. Secondary outcome, perceived differences in quality among plans (NS)

22. Secondary outcome, perceived differences in quality among medical groups (NS)

23. Secondary outcome, self-efficacy (NS)

24. Secondary outcome, perceived benefits and barriers (NS)

25. Secondary outcome, Difficulty of selecting plan (p=0.003)

26. Limitations • Primary outcome (actual choice of health plan) may be difficult to change because of competing concerns (e.g., price, convenience) and information from other sources (e.g., friends and family) • Analysis of secondary outcomes limited by poor response to post-OE survey despite two mailings, financial incentive, and follow-up abbreviated web-based survey (est. 41% excluding ineligibles)

27. Policy implications • Educational/motivational interventions designed to increase perceived benefits and decrease perceived barriers, with negative framing, may increase use of quality information but are unlikely to affect actual choices in the health care market. • Quality data with negative framing may make decision-making more difficult for price-sensitive consumers (especially if there is a perceived cost-quality tradeoff). • Many other signals affect consumers’ behavior during Open Enrollment; cost is the dominant factor in the small business market in the USA.