Approaches for Supporting Evidence- and Values-Informed Policymaking

1. Approaches for Supporting Evidence- and Values-Informed Policymaking Knowledge, Sharing, Doing: 1st National KT Conference in Rehabilitation 4 May 2016 Montréal, Québec Michael G. Wilson, PhD Assistant Director, McMaster Health Forum Assistant Professor, McMaster University

2. Evidence- and Values-Informed Policy (1) • Systematically and transparently using the best available data and research evidence, as well as citizens’ values and preferences, in each of: • Prioritizing problems and understanding their causes (agenda setting) • Deciding which option to pursue (policy development) • Ensuring the chosen option makes an optimal impact at acceptable cost (policy implementation) • … alongside the institutional constraints, interest-group pressure, values and other types of information (like jurisdictional reviews, consultations, expert review groups, and opinion polls) that influence the policy process 2

3. Evidence- and Values-Informed Policy (2) Source: Hoffman SJ, Rottingen J-A, Bennett S, Lavis JN, Edge JS, Frenk J. Building Health Systems Research as a Field of Scientific Endeavour: Wading through Definitional Confusion, Conceptual Challenges and Opportunities for the Future, Manuscript under view. 3

4. Options Available to Support the Use of Research Evidence Two factors that consistently increased the prospects for research use in management / policy (based on a systematic review of 124 studies) Citation: Lavis JN, Catallo C, editors (2013). Bridging the worlds of research and policy in European health systems. Copenhagen, WHO Regional Office for Europe. 4

5. Examples of Approaches 5

6. One-Stop-Shops (1) • One-stop shops (and evidence services) are a promising ‘self-serve’ KT innovation for several reasons • Supports timely access (everything in one place) • Facilitates assessments of relevance • Organized by priority topics (e.g., health systems) • Provide decision-relevant information (e.g., quality, countries in which included studies are conducted) • Enhances communication • Presentation of evidence in several formats and in ways that are user friendly (e.g. links to free full text or user-friendly summaries) 6

7. One-Stop-Shops (2) 7

8. One-Stop-Shops (3) 8

9. Evaluation & Impact • One-stop-shops (Health Systems Evidence) • 11,596 registered users (4403 signed up to receive a monthly customized evidence service) • Researchers = 3874; Policymakers = 2460; Healthcare professionals = 2630; Managers = 1261; Plus more than 3000 students and 2464 ‘other’ • Endorsed by WHO’s Health Systems Research Synthesis Group as the one-stop shop for research syntheses about health systems • Incorporated into other resources (e.g., EVIPNet Virtual Health Library, McMaster Optimal Aging Portal) • Increasingly used to inform high-profile scientific articles / studies • Cited as the key source in the New England Journal of Medicine (Mills) and in Health Policy (Rockers et al.) 9

10. Rapid-Response Units (1) • Often need to address pressing health system issues in days or weeks • May need support with finding and synthesizing research evidence given competing demands (but not enough time to prepare an evidence brief and convene a dialogue) • Rapid-response units fills a gap between • ‘Self-serve’ approaches (e.g., one-stop shops) and • ‘Full-serve’ approaches (e.g., stakeholder dialogues informed by evidence briefs) 10

11. Rapid-Response Units (2) • There are several existing programs in Canada (e.g., CADTH, INESSS, OHTN), but not specifically focused on health-system questions • 11 stakeholder dialogue participants agreed that there is a clear need for such a program • Key recommendations: • build a model • decide what can done in what timelines • define success and measure it 11

12. Evaluation & Impact • Rapid response • Evaluation approach is in development • Examples of impacts • A synthesis about suicide-prevention interventions directly informed the 2014 Toronto Public Health Suicide Prevention Strategy • Three syntheses for the College of Physicians and Surgeons of Ontario directly informed College policies • unsafe medical practices • preventing sexual abuse • educational approaches to improve clinical performance 12

13. Citizen Panels (1) • Key challenges • Complex health problems • different understandings of the problem • Uncertainties • e.g., about the most effective policy options to address these problems and their implementation considerations (e.g., equity, costs, unintended effects, acceptability and feasibility) • Lack of agreement • among all stakeholders about how to move forward 13

14. Citizen Panels (2) • The role of citizens’ values and preferences • Citizens can help us to develop ashared understandingabout a problem(challenge 1) • Citizens have experiential knowledge (challenge 2) • valid and legitimate evidence that can help to find innovative and local solutions to complex problems • Citizens can facilitate or trigger action(challenge 3) • offering guidance on how to move forward • identifying what options are socially, politically, and ethically sound • advocating for policy options 14

15. Citizen Panels & Stakeholder Dialogue Process Overview 15

16. 1. Consulting with Stakeholders • Steering committee • Develop criteria to select diverse panel participants • Iteratively refine understanding of the problem, possible options to address it and implementation considerations • Identify key informants • Key informant interviews • 15-20 interviews with representatives of citizen/patient groups, policymakers, stakeholders and researchers who are involved in or affected by the issue 16

17. 2. Preparing a Citizen & Evidence Brief • Presents (in plain language) relevant research evidence about a problem, options for addressing it, and key implementation considerations • Based on syntheses and local evidence (identified using systematic and transparent approach) • No recommendations • Subjected to merit review • Citizen brief poses questions for citizens to consider 17

18. 3a. Convening a Citizen Panel • Brings together a diverse group of 10 to 16 citizens for a one-day, off-the-record, dialogue that provides them with the opportunity to: • bring their own views and experiences to bear on an issue; • learn from the evidence and from others’ views and experiences; and • share their newly informed views about the issue and how to address it. 18

19. 3b. Preparing a Summary • Thematic analysis of deliberations • Describes: • views about and experiences related to the issue • values and preferences for addressing it • Identify areas of shared understandings, as well as divergent opinions 19

20. 4a. Convening a Stakeholder Dialogue • Brings together 18-22 policymakers, stakeholders and researchers for ‘off-the-record’ deliberations about: • the problem • options to address it • implementation considerations • next steps • Participants chosen because of their ability to: • bring unique views and experiences • champion actions to address the challenge creatively 20

21. 4b. Preparing a Summary • Thematic analysis of deliberations • Identifies areas of common ground • Divergent opinions • Respects Chatham House Rules 21

22. 5. Supporting Action • Two examples of ways to support action • Take an integrated approach to supporting evidence- and values-informed policymaking by convening a citizen panel followed by a stakeholder dialogue on the same topic • informed by an evidence brief that includes findings from the citizen panel • ‘off-the-record’ deliberations with those who are in a position to champion change • Personalized briefings to those who can take action 22

23. Evaluation & Impact (1) • Citizen brief (n=14 briefs; n=211 respondents) • Mean overall assessment = 6.1 (SD = 1.2) • Range of ratings of 14 features of citizen briefs = 5.4-6.4 • Citizen panels (n=20 panels n=215 respondents) • Mean overall assessment = 6.7 (SD = 1.2) • Range of ratings of 11 specific design features = 6.3-6.9 • Self-rated knowledge of the topic increased from pre- to post-panel (mean of 5.1 to 5.6) • Example of impact • Directly informed the Ontario Medical Association’s end-of-life care strategy and the Canadian Medical Association’s national dialogue about end-of-life care 23

24. Evaluation & Impact (2) • Evidence briefs (n=41, n= 610 respondents) • Overall rating of brief = 6.2, all but five features rated ≥ 6.0, and only three features had much variation (SD>1.1) • Dialogues (n=41, n=613 respondents) • Overall rating of dialogue = 6.2, all but one feature rated ≥ 6.1, and no features had much variation (SD>1.1) • Examples of impact • Directly informed an Ontario provincial cabinet submission about creating community-based specialty clinics • Spurred formation of committee reporting to Ontario deputy minister to optimize clinical practice based on data, evidence and guidelines 24

25. Evaluation & Impact (3) • Dialogues (continued) 25

26. Training/Capacity Building • Policymakers, stakeholders and researchers require support to develop skills in finding and using research evidence to inform the stages of policy development (clarifying problems, framing options, identifying implementation considerations) • E.g., - Health Systems Learning • Finding and Using Research Evidence to Inform Decision-making in Health Systems and Organizations • Three course objectives: • To develop knowledge about tools and resources available to help health system decision-makers in order to support their use of research evidence • To examine the attitudes that are supportive of using research evidence in health system decision-making • To enhance skills in acquiring, assessing, adapting and applying research evidence 26

27. Evaluation & Impact Training/capacity building • 1,055 trainees (639 in-person, 190 in-person+online and 226 online) • 531 policymakers, 42 managers, 47 students, 15 professionals, 16 researchers, 13 ‘other’ and 180 in multiple roles completed evaluation (RR=80%) • Strongly positive feedback from evaluations (measured on seven-point Likert scale) • Overall rating = 6.0 (range = 4 - 7) • Highest rated feature: Material relevant to my professional development = 6.6 (range = 3-7) • Lowest ratings: The workshop enhanced local applicability assessment skills = 5.8 (range = 2-7) • 16 of 18 design features had average ratings of 6 or more 27

28. Stay current with updates about Forum-led initiatives, events and training opportunities, via Twitter @MacHealthForum; and Facebook McMaster Heath Forum 28