John N. Lavis, MD, PhD Associate Professor and

1. 3 December 2004 Enhancing the Uptake of Research Knowledge in Health Policy Leonard Davis Institute of Health EconomicsUniversity of Pennsylvania, Philadelphia, PA, USA John N. Lavis, MD, PhD Associate Professor and Canada Research Chair in Knowledge Transfer and Uptake McMaster University

2. Overview • Recognizing we can only make small differences • (the big differences are usually just plain luck) • Doing what we can do (differently) • Producer-push approach • User-pull approach • Knowledge-exchange approach • Helping those who want to do it differently

3. We Can Only Make Small Differences • What is the place for research knowledge in health policy? • Helps to get problems on the government agenda • Helps to solve particular problems at hand • Helps to think about problems and solutions differently • Helps to justify a decision made for other reasons • How can the uptake of research knowledge in health policy be enhanced? • Identify and take action on the modifiable aspects of complex knowledge transfer, uptake & exchange processes and complex decision-making processes

4. Doing What We Can Do (Differently) • Embrace the best of each of three approaches • Producers and purveyors of research actively push research knowledge out to users (producer/purveyor-push efforts) • Users of research actively pull in research when faced with a decision that they believe could be informed by research knowledge (user-pull efforts) • Producers and users of researcher are jointly responsible for transferring and facilitating the uptake of research (and other forms of) knowledge (knowledge-exchange efforts)

5. Producer/Purveyor-Push Efforts • Producers and purveyors of research can more effectively transfer and facilitate the uptake of research knowledge by re-thinking • What should be transferred? • To whom should it be transferred? • By whom should it be transferred? • How should it be transferred? • With what effect should it be transferred?

6. Producer/Purveyor-Push Efforts (2) • What should be transferred? • Transfer actionable messages based on bodies of research knowledge, not just individual studies, and seek to achieve consensus on the messages • Translation # 1 = create music, not noise • Translation # 2 = focus knowledge-transfer efforts at the apex of the knowledge pyramid while continuing to build a solid base for the pyramid

7. Producer/Purveyor-Push Efforts (3) • Actionable messages • Syntheses of research knowledge • Individual studies, articles, and reports • Basic, theoretical and methodological innovations

8. Producer/Purveyor-Push Efforts (4) • To whom should it be transferred? • Identify the most appropriate target audience(s) for each message and fine-tune the message and approach to knowledge transfer for each target audience • Possible target audiences include: • General public and civil society groups • Patients • Clinicians • Health-system managers • Employers • Public policymakers (i.e., politicians, civil servants, and political staffers in line departments and central agencies)

9. Producer/Purveyor-Push Efforts (5) • By whom should it be transferred? • Identify and work with or through the most credible messenger for each combination of message and target audience • Messengers may be researchers, “knowledge brokers” or opinion leaders drawn from the target audience • How should it be transferred? • Employ proven approaches to transferring and facilitating the uptake of the messages, almost all of which involve some type of face-to-face interaction (Grimshaw 2001; Innvaer et al. 2002) • And evaluate promising approaches, such as the European Observatory’s “rapid reaction evidence seminars”

10. Producer/Purveyor-Push Efforts (6) • With what effect should it be transferred? • Evaluate the impact of producer-push efforts against appropriate objectives and share experiences with others • Changing clinical practice can be seen as an appropriate objective when an intervention is clearly unsafe or ineffective • Informing political debate, not successfully changing or introducing a public policy, can be seen as an appropriate objective in a democracy

11. User-Pull Efforts • Users of research can be more effectively supported to use research knowledge in their decision-making by developing • Capacity to use research knowledge (CHSRF 2000) • Identify and access relevant research (acquire)*** • Critically appraise its quality and local applicability (appraise)*** • Adapt it so that it can be understood locally (adapt) • Apply it (apply) • Capacity to commission a systematic review (or research) when none exists • Identify gaps in existing research knowledge • Mobilize health-research systems to fill the gaps

12. User-Pull Efforts (2) • Acquiring research - challenges • Lots of questions • Finding effective and cost-effective solutions to the most burdensome health problems • Fitting these solutions into health systems • Bringing about change in health systems • Many types of evidence can inform answers to these questions • Letters and calls from disease groups • Support or opposition by professional associations • Experiences of neighbouring jurisdictions • Research-based evidence • So what’s the most efficient way to acquire it?

13. User-Pull Efforts (3) • Acquiring research - opportunities • Systematic reviews of research-based evidence • Reduce the likelihood that public policymakers will be misled by research (by being more systematic and transparent in the identification, selection, appraisal & synthesis of studies) • Increase confidence among public policymakers about what can be expected from an intervention (by increasing the number of units for study) • Allow civil servants and political staff to focus on appraising the local applicability of syntheses and on collecting and synthesizing other types of evidence, such as political acceptability and feasibility – i.e., allow them to focus on the apex of the research knowledge pyramid while doing the rest of their jobs

14. User-Pull Efforts (4) • Assessing research - challenges • Context matters in health systems • Commonalities in human biology mean that a prescription drug will often work the same way in different people • Differences in health systems mean that an intervention that works in one jurisdiction may not work the same way in another jurisdiction, and systematic reviews may not contain studies that were conducted in a public policymaker’s jurisdiction • So how can a public policymaker navigate between the extremes of assuming no transferability of a systematic review when no research from their jurisdiction was included in the review and assuming the full transferability of a systematic review?

15. User-Pull Efforts (5) • Assessing research - opportunities • Checklist for assessing the local applicability of reviews (i.e., what can be expected if same thing is done in our jurisdiction?) • Could it work in my jurisdiction? • Are there important differences in structural elements of health systems that mean an intervention could not work in the same way? (e.g., concentrated ownership) • Will it work? • Are there important differences in the perspectives and influence of stakeholders that mean an intervention is not likely to be taken up in the same way? (e.g., medical associations) • Does the health system face other challenges that substantially alter the potential benefits and harms (or risks) of the intervention? (e.g., infrastructure or HHR, starting points in terms of problem or intervention use)

16. User-Pull Efforts (6) • Assessing research – opportunities (2) • Checklist for assessing the local applicability of reviews (2) • What would it take to make it work? • Can power dynamics and on-the-ground realities and constraints be changed in the short- to medium-term and what are the prospects for making this happen? • Is it worth it? • Is the balance of benefits and harms (or risks) classifiable as net benefits, trade-offs (between different types of benefits), uncertain trade-offs, or no net benefits? • Are the incremental health benefits from incorporating the intervention among the mix of interventions provided worth the incremental costs?

17. User-Pull Efforts (7) • Assessing research – opportunities (3) • Systematic reviews and process evaluations can inform how to do things differently (i.e., what can be expected if things are done differently?) • If a public policymaker can only do one or some components of an intervention, which component(s) should be selected? (e.g,. TB DOTS) • If a public policymaker is committed to undertaking all components of an intervention in the long run but only one or some components can be undertaken now, how should the components be sequenced? (e.g., primary care reform)

18. User-Pull Efforts (8) • Supporting policymakers - challenges • Like many countries, Canada and the United States have only weak institutional mechanisms to support public policymakers to acquire, assess, adapt and apply syntheses • Lack of peer review and ethical review requirements for reviews as a pre-condition for project funding (as in the UK) and presence of incentives to “push” individual studies • Lack of processes that bring the producers and users of reviews together to inform the production and adaptation of syntheses (as in the UK and to some extent in the US), the identification of actionable messages, and the development of evidence-based knowledge-transfer activities • Lack of a country-wide license for one-stop shopping (through The Cochrane Library) for quality appraised Cochrane reviews, other types of systematic reviews, and economic evaluations (as in Australia, SA, Spain, UK, etc.)

19. User-Pull Efforts (9) • Supporting policymakers – challenges (2) • Canada and the United States have only weak institutional mechanisms to support public policymakers to acquire, assess, adapt and apply syntheses (2) • Lack of a support function for public policymakers that meets their needs and timelines (as in Norway, to some extent in Europe, and possibly soon in Africa) • Within hours - identification of syntheses on relevant dimensions of the topic • Within days - assessment of the quality and local applicability of identified syntheses • Within months - conduct a systematic review (subject to capacity constraints)

20. User-Pull Efforts (10) • Supporting policymakers - opportunities • Canada and the United States have the building blocks for such institutional mechanisms in place • Voluntary, investigator-led reviews of health-systems research are being coordinated by Cochrane’s Effective Practice and Organization of Care (EPOC) review group • Processes are in place (at AHRQ) or are being developed (at the Canadian Health Services Research Foundation) to bring the producers and users of syntheses together to inform the production & adaptation of systematic reviews (& to develop capacity to use them) • Canadian Institutes of Health Research have floated the idea of Centres in Health Innovation and AHRQ has the User-Liason Program

21. User-Pull Efforts (11) • Two final observations • Some researchers will tell you that research funders shouldn’t support the funding of systematic reviews • Reviews produce new knowledge and so they are a legitimate and important part of the research enterprise • Some researchers will tell you that Cochrane reviews are for clinical interventions, not health-system interventions • EPOC reviewers do not limit themselves to randomized controlled trials in answering the question “what works?” (they do limit themselves to designs suited to answering that question) • Other types of systematic reviews are helpful to support how to think about problems and solutions differently

22. Knowledge-Exchange Efforts • Producers and (representatives of the) users of research can work towards the creation of a more research-attuned culture among the users of research and a more decision-relevant culture among the producers of research by investing in long-term knowledge-exchange relationships with one another that involve • Shared responsibility for transferring and facilitating the uptake of research (and other forms of) knowledge • Mutual respect for the knowledge that different people bring to the table • Development of jointly “owned” knowledge about how to improve health systems and achieve health goals

23. Helping Those Who Want To Do It Differently • Producers and purveyors of research • If you have the skills and inclination to engage in producer/purveyor-push efforts, consider re-thinking your answers to the five questions • If you have the skills, inclination, and patience to engage in knowledge-exchange efforts, identify and develop a long-term knowledge-exchange relationship with a decision-maker partner • If you have the skills, inclination, patience, and fortitude to help us get away from the relentless media coverage of individual studies regardless of their marginal contribution to research knowledge, consider taking steps to educate the media about the advantages of focusing on the “release” of actionable messages based on bodies of research knowledge

24. Helping Those Who Want To Do It Differently (2) • Users of research • If you have the skills and inclination to engage in user-pull efforts, assess your current capacity to use research knowledge and to commission a systematic review (or research) when none exists, and take steps to build capacity in areas where room for improvement can be found • If you have the skills, inclination, and patience to engage in knowledge-exchange efforts, identify and develop a long-term knowledge-exchange relationship with a research partner • (Ditto re the media)

25. Helping Those Who Want To Do It Differently (3) • Funders of research • If the current funding situation in your state or country allows you to focus some of your resources on the apex of the knowledge pyramid while knowing that its base is not being jeopardized, create capacity, appropriate rewards and accountability for those who have the skills and inclination to to employ effective producer/purveyor-push, user-pull and knowledge-exchange efforts • Stop funding dissemination outside the scholarly community as a key element of every research project • Launch processes that bring the producers and users of research together to develop consensus on actionable messages and lead initiatives to facilitate their uptake • Launch capacity-building initiatives

26. References • World Health Organization. (2004) World Report on Knowledge for Better Health. Geneva: WHO. [http://www.who.int/rpc/meetings/pub1/en/0] • Grimshaw J, Shirran L, Thomas R et al. (2001). Changing provider behaviour: An overview of systematic reviews of interventions. Medical Care 39(Supplement 2):II2-II45. • Innvaer S, Vist GE, Trommald M, Oxman AD (2002). Health policy-makers’ perceptions of their use of evidence: A systematic review. Journal of Health Services Research and Policy 7(4):239-244.

27. References (2) • Lavis JN, Robertson D, Woodside JM, McLeod CB, Abelson J and the Knowledge Transfer Study Group (2003a). How can research organizations more effectively transfer research knowledge to decision-makers? The Milbank Quarterly 81(2):221-248. • Lavis JN, Ross SE, McLeod CB, Gildiner A (2003b). Measuring the impact of health research. Journal of Health Services Research and Policy 8(3):165-170.

28. References (3) • Lavis JN, Ross SE, Hurley JE, Hohenadel JM, Stoddart GL, Woodward CA, Abelson J (2002). Examining the role of health services research in public policymaking. The Milbank Quarterly 80(1):125-154. • Lavis JN, Becerra Posada F, Haines A, Osei E. (2004) Use of research to inform public policymaking. The Lancet 364:1615-1621. • Canadian Health Services Research Foundation (2000). Self Assessment Tool for Decision-Making Organizations. Ottawa: Canadian Health Services Research Foundation.

29. Contact Information • John N. Lavis • lavisj@mcmaster.ca • Program in Policy Decision-Making, McMaster University • www.researchtopolicy.ca