Lost in Translation: How Do We Increase the Return on Investment from Research?

1. Lost in Translation: How Do We Increase the Return on Investment from Research? Lisa Simpson, MB, BCh, MPH, FAAP 18th Annual National Maternal and Child Health Leadership Conference May 16, 2005

3. Outline • Some fundamental assumptions • The translation imperative • What do we know about what promotes – or impedes - research translation? • Unique issues in MCH practice • Measuring Impact – the return on investment • Strategies for MCH practice

4. Some Fundamental Assumptions • Research translation will result in more informed and more effective policies and practice • Policymaking encompasses at least • Public/legislative • Administrative/systems • Clinical/practice • Use is NOT in of itself impact • This is not a new problem!

5. The Case of Scurvy • 1601: Lancaster shows that lemon juice supplement eliminates scurvy among sailors • 1747: Lind shows that citrus juice supplement eliminates scurvy • 1795: (194 years after discovery) British Navy implements citrus juice supplement Source: Mosteller, Science l981;221:881

6. The Case for Prenatal Corticosteroids • 1969: Liggins original research • 1974-1995: 17 trials conducted • 1989: first systematic review • 1990’s: continued underuse between 20-30% • 1995: NIH consensus conference • 1999: Leviton et al. trial shows increased use from 33% to 58% (passive dissemination) or 68% (active dissemination) Hanney et al, Social Science & Medicine, 2004

7. Balas, 2002 Original research 18% variable Negative results Dickersin, 1987 Submission 46% 0.5 year Kumar, 1992 Koren, 1989 Acceptance Negative results 0.6 year Kumar, 1992 Publication 17:14 35% 0.3 year Poyer, 1982 Balas, 1995 Lack of numbers Bibliographicdatabases Expert opinion 50% 6. 0 - 13.0 years Antman, 1992 Poynard, 1985 Reviews, guidelines, textbook 9.3 years Inconsistent indexing Implementation It takes 17 years to turn 14 per cent of original research to the benefit of patient care

8. The Translation Imperative • Growing chorus nationally and internationally • Focus of World Health Report 2004 • EBM movement is itself generating pressure and a call for EBPP: evidence based policy & practice • Governmental accountability: GPRA • Growing recognition of the scope and severity of the healthcare quality challenges • Doubling the budget of the National Institutes of Health

13. Investments in Children: NIH Budget • Overall doubling between FY 1998 & 2003 • Pediatric spending increased at an average annual rate of 12.8% • Proportion of NIH budget devoted to pediatric portfolio from 12.3% to 11.3% • Total pediatric portfolio = $3.2 billion (FY 2004) • 2/3 of portfolio in just 5 of 22 IC’s: NICHD, NIMH, NIDDK, NHLBI, and NCI Gitterman, Health Affairs, 2004

14. Evolution of the Social Contract Public funding in exchange for high quality research What is the return on investment? What is the utility of research? Who is using the research and what difference does it make? Adapted from Rich, 2002

15. Translational Blocks • Lack of Willing Participants • Regulatory Burden • Fragmented Infrastructure • Incompatible Databases • Lack of Qualified Investigators • Career Disincentives • Practice Limitations • High Research Costs • Lack of Funding 1 2 Translation of New Knowledge Into Clinical Practice And Health Decision Making Translation From Basic Science To Human Studies Basic Biomedical Research Clinical Science And Knowledge Improved Health The Two Translational Blocksin the Clinical Research Continuum Clinical Research Continuum IOM Clinical Research Roundtable, Sung et al, JAMA, 2003

16. Translational Blocks • Lack of Willing Participants • Regulatory Burden • Fragmented Infrastructure • Incompatible Databases • Lack of Qualified Investigators • Career Disincentives • Practice Limitations • High Research Costs • Lack of Funding 1 2 Translation of New Knowledge Into Clinical Practice And Health Decision Making Translation From Basic Science To Human Studies Basic Biomedical Research Clinical Science And Knowledge Improved Health The Two Translational Blocksin the Clinical Research Continuum Clinical Research Continuum IOM Clinical Research Roundtable, Sung et al, JAMA, 2003

17. Translational Blocks • Lack of Willing Participants • Regulatory Burden • Fragmented Infrastructure • Incompatible Databases • Lack of Qualified Investigators • Career Disincentives • Practice Limitations • High Research Costs • Lack of Funding 1 2 Translation of New Knowledge Into Clinical Practice And Health Decision Making Translation From Basic Science To Human Studies Basic Biomedical Research Clinical Science And Knowledge Improved Health The Two Translational Blocksin the Clinical Research Continuum Clinical Research Continuum IOM Clinical Research Roundtable, Sung et al, JAMA, 2003

18. Outline • Some fundamental assumptions • The translation imperative • What do we know about what promotes – or impedes - research translation? • Unique issues in MCH practice • Measuring Impact – the return on investment • Strategies for MCH practice

19. Rogers’ Diffusion S-Curve Laggards Late majority Early majority Adoption Early adopters Innovators Time

20. Hierarchy of Technology Adopters

21. Wise Adaptation of Rogers’ S-Curve Pop.A Pop. B Laggards Late majority Early majority Adoption Pop. C Early adopters Innovators Time

22. The Dynamics of Innovation Diffusion • The Innovation/New knowledge (message) • Communication Channels (medium & messenger) • Social context

23. Diffusion of Innovations • The Innovation/New knowledge • Relative advantage • Ability to judge if benefits outweigh risks. • Interplay between interests of patient, clinician & system • Does not occur when controversy exists about findings – e.g. mammography. • LESSONS: • Understand end user • Recognize impact of change • Consider business case • Use testimonials & success • stories

24. Diffusion of Innovations • The Innovation/New knowledge • Relative advantage • Trialibility • Ability to test out new knowledge/innovation without total commitment & with minimal investment • Reduces uncertainty about risks & benefits • LESSONS: • Use small tests of change • Try out in one program • Focus on knowledge & • innovations that can be tested

25. Diffusion of Innovations • The Innovation/New knowledge • Relative advantage • Trialibility • Observability • Ability to watch others applying knowledge or using the innovation • Better evidence of improved decisionmaking, increased functionality, and better outcomes • LESSONS: • Bring groups together to share experiences • Use charismatic opinion leaders to demonstrate success • Use “viral marketing”

26. Diffusion of Innovations • The Innovation/New knowledge • Relative advantage • Trialibility • Observability • Compatibility • Compatible with values, past experiences and needs of user • Addresses an issue which users agree is a problem • LESSONS: • Look for current behaviors/practices that are similar to the one being introduced • Innovations that reduce hassles are more likely to be successful

27. Diffusion of Innovations • Communication Channels (medium) • Internet & medical literature and new online journals with discussion portals • Web based affinity groups (NICHQ collaboratives) • Direct to consumer information • Patient mediated clinician/system change • Personal contact • “Piggy-backing” • LESSONS: • Methods to inform vs. persuading are different • The more complex the message, the more face-to-face is important • Match channel to audience & innovation • Use “connectors” or “information brokers”

28. Diffusion of Innovations • Social context • Powerful driver of behavior – “ that’s how I learned to do it” • Code of silence among physicians still hindering quality and safety movement • Important to choose the right network for diffusion – e.g. of practice guidelines • Many barriers at the financing/regulatory level • LESSONS: • Leverage existing social & professional networks • Leadership, a commitment to excellence and an organizational culture of creativity essential • Identify the system barriers to adoption

29. Factors that influence policy application of research • Pertinence [relevance] • Ideological acceptability • Practicality • Issue complexity • Time urgency • Power and interest group politics • Method of transmission: messenger and message frame Adapted from Davis & Howden-Chapman, 1996

30. Policymakers Views on Research Use • 24 studies with 2041 policymaker interviews • Facilitators: • Personal contact (54%) • Timeliness & relevance (54%) • Summary & recommendations included (46%) • Good quality (25%) • Confirmed current policy or endorsed self interest (25%) • Community pressure (17%) Innvaer et al, J Health Serv Res Policy, 2002

31. Policymakers Views on Research Use • Barriers: • Lack of personal contact (46%) • Lack of timeliness or relevance (38%) • Mutual mistrust and reciprocal naivete (33%) • Power & budget struggles (29%) • Poor quality (25%) • Political instability (21%) Innvaer et al, J Health Serv Res Policy, 2002

32. Policymakers on Researchers • They take forever to answer even the simplest question(s). Then they’re late with the results. • What is this stuff they write? Who can understand it? • They work in their “ivory towers” and produce impractical or irrelevant ideas and recommendations. • They’re always hedging. I can’t get a straight answer. • They don’t answer the question I thought I asked • They take little responsibility for the implications of their findings.

33. Researchers on Policymakers • They don’t ask researchable questions. • They don’t accept uncertainty. They don’t accept that reducing uncertainty costs more money. • They don’t appreciate the influence of “publish or perish” on my life. • They want unrealistic turn-around for results. • They expect me to drop everything and deliver results for policy input. • They want “bottom line” answers to take them off the policy hook. • They can’t be trusted with my results - may misinterpret or misuse my results.

34. From ‘Push’ to ‘Push, Pull and Partner’ Implications: • attention to dissemination + receptor capacity • ongoing interaction between the two processes ‘linkage and exchange’ • knowledge brokers/boundary spanners (Lomas, CHSRF)

35. Information Brokers • Government agencies, e.g. AHRQ, MCHB • Foundations, e.g. Commonwealth Fund • Research and policy centers • University, not-for-profit & for-profit • EIS • Constituent organizations, e.g. AMCHP, ASTHO

36. Change Agents • “an individual who influences clients’ innovation decisions in a direction deemed desirable by the change agency” • Rogers, 2003 • Academic detailing: • Almost 60,000 pharmaceutical sales representatives – one for every 14 doctors!!

37. Strategies to Promote Clinical Translation • Multiple reviews in last ten years • Prevailing wisdom • Multifaceted interventions more effective • Passive dissemination (e.g. CME ineffective) • Most recent review • >200 studies • Mean absolute improvement of only 10% • Wide variation in effect size (-1% to 34%) • Multifaceted interventions not significantly more effective than single faceted ones • Passive dissemination produced modest, but consistently positive improvements Shojania & Grimshaw, Health Affairs, 2005

38. Outline • Some fundamental assumptions • The translation imperative • What do we know about what promotes – or impedes - research translation? • Unique issues in MCH practice • Measuring Impact – the return on investment • Strategies for MCH practice

39. Stakeholders in Children’s Healthcare Quality Clinical Policy Decisionmakers children, teens, families, practitioners Health Care Systems Decisionmakers health plans, hospitals, CHC’s, LHDs, MCH Programs, schools Public Policy Decisionmakers State, Federal, Voluntary

40. Types of Decisions • Public policy Do we reimburse for ADHD related assessments? • Systems policy How do we promote better ADHD identification and management? • Clinical policy (“EBM”) Which children should be managed with which strategies?

41. Unique Barriers in MCH Practice • Dominance of private sector, market based approach: knowledge as competitive advantage • Skepticism as to the role of government • Prevailing view of children as responsibility of parents • Fragmented systems of care for children • Heavy reliance on state action for children’s health & wellbeing multiple & diffuse receptor sites

42. Unique Barriers in MCH Practice • Multiple levels of policy decisionmaking • 50 Medicaid program • 50 Title V programs • 35 Stand alone SCHIP programs • Multiple sites of care • Clinical settings • Health departments • Schools • Early intervention programs

43. Unique Barriers in MCH Practice • Under resourced providers • Child health providers: lowest paid, undercapitalized • Under investment in translation infrastructure • Improvement collaboratives • Information technology applications • Return on investment (ROI) more difficult

44. Exploring the ROI for Quality in Children’s Health Care • A business case for health care improvement exists if… • the investing entity realizes a financial return on investment in a reasonable time frame & using a reasonable discount rate • “bankable dollars”, reduction in losses, or avoided costs are realized • the investing entity believes an important indirect effect on organizational functioning and sustainability would accrue in a reasonable time frame (e.g. increased market share, staff retention) Leatherman et al, Health Affairs, 2003

45. Barriers to the Business Case for Quality in Health Care • Inability of consumers to perceive quality differences • quality information even less available in child health • Displacement of payoffs in time & sector • insurance churning; benefit reaped in schools • Disconnect between consumers & payers • Failure to pay for quality while paying for defects • Task-based fee schedules with no incentive for quality • Uneven access to information among clinicians • Evidence base for children’s health less robust Modified from Leatherman et al, Health Affairs, 2003

46. Outline • Some fundamental assumptions • The translation imperative • What do we know about what promotes – or impedes - research translation? • Unique issues in MCH practice • Measuring Impact – the return on investment • Strategies for MCH practice

47. Government likes to begin things – to declare grand new programs and causes and national objectives.  But good beginnings are not the measure of success.  Government should be results-oriented – guided not by process but guided by performance.  There comes a time when every program must be judged either a success or a failure. George W. Bush, January 2000

48. Program Results • Has the program demonstrated adequate progress in achieving its long-term outcome goal(s)? • Does the program (including program partners) achieve its annual performance goals? • Does the performance of this program compare favorably to other programs with similar purpose and goals? • Do comprehensive, independent evaluations of this program indicate that the program is effective and achieving results?

49. Hierarchy of Research Impact Improves access, outcomes, quality Level 4 Improves delivery and practice Level 3 Level 2 Improves process and policies Level 1 Improves other research Stryer, Tunis & Clancy, 1999

50. Levels of Impact:the Case of ADHD