“Puffery”

1. “The public is concerned about hospital quality but has less information about selecting a hospital than they do when purchasing a toaster or a car” (Malone, 1999)

2. “Puffery” • “…representation, statement, or conduct that clearly over-exaggerates… • …is not intended to be an offer to be relied upon.” • Consumer protection (2000 Legal on-line Dictionary)

4. Methods • Solucient’s “Top 100 Hospitals” • US News & World Report’s “Best Hospitals” • American Nurses Credentialing Center’s “Magnet Recognition Program” • Descriptive statistics • Spatial mapping • Review of empirical literature

5. US News & World Report’s “Best” Hospitals “If you are looking for the ‘best’ in medical care…” —web-based guide. “designed to assist you in your search for the highest level of medical care.” (US News & World Report)

6. US News: Selecting the “Best” • National Opinion Research Center • Each July since 1990 • Ranked by specialties (17) • “Only rigorous assessment of all US hospitals” • Eligible -- “tertiary care hospitals” • COTH membership or • Affiliation with medical school or • Availability of 9 out of 17 prescribed technologies • Hospital Quality Index (HQI) • Honor Roll – score well in at least 6 specialties

7. US News continued… • Measures: produce HQI • Structure = AHA Annual Survey data • FTE “on-staff” Nurse to bed staffing ratios • Services available (technology index) • # of specialty-specific technology services • Process = (“proxy measure”) • Physician nominations • Newest specialties reputation only • Outcomes = • Medicare risk-adjusted mortality by specialty • Pool most recent 3 years • Mortality ratio = # inpatient deaths/expected

8. Solucient’s “Top 100” Hospitals • “Identifies industry benchmarks by recognizing hospitals that demonstrate superior clinical, operational, and financial performance.”

9. Selecting the “Top 100” • Since 1993 -- December • “Management results that separate these hospitals from the pack.” • “Which hospitals offer superior care at lower costs.” • “Data obsession--common trait.” • Enterprise = benchmark reports for purchase • Eligibility restricted: 5 categories (n=20) • Overall + select specialties (ICU, CV, Ortho, Stroke benchmarks) • Data source – Medicare Cost Report

10. Solucient “TOP” – 5 strata • Major Teaching (400 or more beds & resident to bed ratio of at least .25) • Teaching (at least 5 residents or resident to bed ratio of .01-.24) • Large Community (250 or more beds) • Medium Community (100-249 beds) • Small Community (25-99 beds)

11. “Top 100”selection… • Ranked by strata on following measures—20 best rankings per strata selected • Financial management = • Expense/adjusted discharge: case mix and wage adjusted • Profitability • Proportion of outpatient revenues • Index of total facility occupancy • Clinical performance = • Medicare risk-adjusted mortality index • Risk-adjusted complication index • Operational performance = • Severity-adjusted average LOS • Case-mix and wage-adjusted expense per discharge

12. ANCC Magnet Recognition Program for Excellence in Nursing (1994)

13. ANCC Recognizes “excellence” in 4 target areas • Management philosophy & practices • Adherence to standards for improving the quality of care • Leadership of the CNO in supporting professional practice and continued nursing competence 4. Attention to the cultural and ethnic diversity of patients, their significant others, and providers

14. Magnet Recognition Process • Self evaluation: ANA’s Scope & Standards • ANA quality indicator data (NDNQI) • Site visit • Community input • Could range from $30,000 - $75,000 • Reevaluated every 4 years or more often with leadership change Continuous Process: submit data annually

15. Diffusion of “Magnetism”by year and total number recognized

16. “Creep” & Overlap - 2001 & 2003

17. Best, Top, & Magnet • University of CA Irvine • Poudre Valley Hospital, Fort Collins CO • Mayo Clinic, Rochester MN • NC Baptist, Wake Forest Un., NC • Methodist Hospital, Houston, TX • Inova Fairfax Hospital, Falls Church VA

18. Summary – Best, Top & Magnet

19. U.S. News “Best Hospitals” - 2003

20. Solucient’s “Top 100 Hospitals” - 2003

21. ANCC’s Magnet Hospitals - 2003

22. Best, Top, & Magnets - 2003

23. Regional dominance = “Pockets of excellence” ?? • High preponderance – northeast, south • 6 states – “empty” • Mountain, west north central, west south central regions • Some states in highly designated regions had fewer hospitals than those in the “empty regions” • CA -- 450 hospitals (3 magnets) = .7% • NJ – 74 hospitals (14 magnets) = 19%

24. The US NEWS “Best” -- 3 Empirically based publications • Different rankings when different decision models used (Teasley, 1996) • CABG outcomes in PA and NY (Hartz et al., 1997) - “Best did not perform better than rest” • Significantly lower AMI mortality rates (Chen et al. (1999) • Clinical practice processes • Asprin & beta blockers • Numerous reports of methodological “flaws” & conceptual issues • What is examined • Who does the nomination • “best colleges” • “best party schools”

25. Solucient “Top” -- 3 empirically based publications • CHEN et al., 1999 • AMI patients over 65 Significantly lower LOS and costs than “rest” • No significant difference in mortality rates • Griffith & Alexander 2002 • Found that 8 of the Solucient measures of hospital performance were valid and reliable • Griffith, Knutzen & Alexander, 2002 • “a disjuncture” between “Top” outcome measures and JCAHO evaluations in non-federal hospitals

26. ANCC Magnet Hospitals – 4 Empirically based publications • Aiken, Havens, & Sloane, 2000 • support for nursing practice • nurse reported quality • nurse satisfaction • better nurse staffing ratios • more highly educated nurses • nurse burnout • Havens, 2001 • strong departments of nursing • quality ratings • organizational support for nursing practice • difficulty recruiting nurses • patient and family complaints

27. Upenieks, 2002, 2003 • nurse satisfaction in ANCC magnet hospitals -- linked to stronger nurse manager leadership • levels of RN reported empowerment • Havens, (Forthcoming) • use of “outside” nurses • vacancy rates

28. Comparison of JCAHO HospitalEvaluation Score Means (2002 JCAHO file)

29. Overall Evaluation ScoresComparative Data - 2002

30. Do JCAHO scores contribute to designation? • No statistically significant differences between designated hospitals and the rest on JCAHO scores (t-test) • Logistic regression models of designation (does JCAHO score predict designation or not?) • Unrelated to receiving best, top, or magnet • Higher JCAHO scores only slightly (and not significantly) predict being designated. • Model fit is poor. Havens, Vasey, & Kellogg 2003, Unpublished

31. What is the take home? • Concern about the quality of hospital care • Adverse events and staffing deficiencies cause alarm • Scant information to assess performance • No universal, “consumer friendly” measure of quality • Growing # of ratings / rankings • Competition and marketing campaigns

32. AHRQ advice re: selecting quality care… • “Look for a hospital that: • Is accredited by JCAHO. • Is rated highly by the state, consumer groups, or other organizations.” http://ahrq.gov/

33. “PR Bonanzas” • “Americans have a love affair with rankings.” • “Public relations goldmine” • Marketing campaigns • News media pick up • Web-sites • “Get a 2nd opinion from the ‘best’ hospitals” • “Helps our development efforts” • US News – “hottest marketing tool in the healthcare industry” • Financial boon for awarding organizations… • And ….more

34. Research Implications • Paucity of research • Mixed findings • Are the “Best really better than the rest”? • Are some of the best better than other bests? • Best on what? • Is this meaningful to consumers - The big picture paradigm? • Reliance on structure and process only • What are the key questions? • Safety • Patient perspectives

35. Policy Implications • A case for moving research to inform policy • Consumer info, mis-info, or dis-info? • As competition grows—release of comparative information will increase • “Desire to hold healthcare organizations accountable…need accurate measures.” (Devers) • How to uniformly gauge quality? • Some argue that public does not understand or will not use…the public is requesting • “top 10” is part of American culture • How to depoliticize – big $ maker for magazines • Use of e-health sites is growing • Being touted as a decision guide for consumers • Web sites • Ladies Home Journal • Reader’s Digest

36. We are on the job! Stay tuned