Using Evidence in Your Work* From Evidence to Action A CIHR Funded Project

1. Using Evidence in Your Work*From Evidence to ActionA CIHR Funded Project *Based on a presentation for the National RAI Forum:"Making the Most of It“: Using Evidence in Your Work, by Sarah Bowen, Ph.D. (Winnipeg, MB, May 12, 2006)

2. WHAT IS EVIDENCE? Information that comes closest to the facts of a matter • the form it takes depends on the context or nature of the problem Findings of high-quality, methodologically appropriate research are most accurate evidence • but because research is often incomplete and sometimes contradictory or unavailable, other kinds of information arenecessary supplements to or stand-ins for research The evidence base for a decision - multiple forms ofevidence combined to balance rigour with expedience—while privileging the former (CHSRF)

3. Values Pragmatics & Contingencies Political Judgment Scientific Evidence Lobbyists & Pressure Groups Resources Professional Experiences & Expertise Habits & Tradition From: CHSRF Conceptualizing and Combining Evidence for Health System Guidance

4. EVIDENCE-BASED OR EVIDENCE-INFORMED? • Evidence-based: roots in quantitative research, particularly systematic reviews, clinical trials • Evidence-informed: recognizes other factors affecting decision-making such as an incomplete evidence base

5. IS SOME EVIDENCE BETTER?A humorous look at hierarchies Class 0: Things I believe Class 0a: Things I believe despite the available data Class 1: Randomized controlled clinical trials that agree with what I believe Class 2: Other prospectively collected data Class 3: Expert opinion Class 4: Randomized controlled clinical trials that don’t agree with what I believe Class 5: What you believe that I don’t • Levels of Belief: From Shaughnessy & Slawson, 2004

6. IS SOME EVIDENCE BETTER? Cont. • Quantitative concept is dominant in health • Emphasis is on experimental & clinical evidence (e.g. Cochrane Collaboration) • Hierarchies of Evidence • systematic reviews & meta-analyses • well-designed Random Control Trials • well-designed non randomized studies • well-designed multi centre non-experimental studies • expert opinion, descriptive studies

7. HIERARCHY CONCEPT LIMITS EVIDENCE-BASED HEALTH SERVICES • Important perspectives of qualitative and mixed methods devalued • Difficulties achieving successful evidence-base for planning and decision-making may overwhelm the importance of doing so • Diminished recognition of importance of “context” for transferability of findings

8. LEVELS OF EIDM

9. CHALLENGES DIFFER DEPENDING ON LEVEL OF DECISION • Clinical • What is “best” practice? • Implementing guidelines & standards • Program • Design (What does a program look like?) • Program Implementation • Policy direction • Getting an issue “on the agenda” • Informing a policy trajectory

10. BARRIERS TO EVIDENCE USE Some barriers differ according to level: • Gap between research & practice • Differences between clinical & management: • Culture • Type of evidence • Type of decisions, decision-making process BUT there are a number of common barriers....

11. Crisis Management culture COMMON BARRIERS: Absence of appropriate structures & processes Lack of confidence and authority to implement change Inadequate relevant, high quality information Difficulty accessing evidence Lack of education opportunities (research literacy) • Resource availability • Skills • Time Lack of knowledge of the value of evidence Lack of senior level support (incl. knowledge of managers) Competing & conflicting demands

12. External (hard to modify) Political context Funding decisions Resource allocation External requirements Crisis management culture Lack of information Perverse incentives Internal (modifiable) Team/org culture or structure Leadership Competing priorities Research “literacy” Acceptance of inevitability of crisis management culture Resistance to change Insufficient time allocated BARRIERS ARE NOT JUST EXTERNAL

13. What does it mean that there is “not enough time”? • Not an organizational priority? (“there is always time for the important things”) • Resource allocation • Aligning resources and processes with stated values and goals • Viewing as an “add-on” • Not an individual priority? (“how can you have enough time to do it over if you don’t have enough time to do it right?”)

14. ORGANIZATIONAL/UNIT/TEAM FACTORS • Lack of mechanisms for consultation and input • Centralized decision-making • Leadership style • Barriers to information flow • Lack of transparency & accountability of decision-making processes • Lack of processes & structures to allow “reflection” time

15. WHAT WORKS IN PROMOTING USE OF RESEARCH EVIDENCE? • Collaboration • Relationships • “Personality factor” • Leadership commitment to EI change • Resource availability • Willingness to change processes • Sustainable interventions Decisions AND implementation

16. USING EVIDENCE IN YOUR WORK • Determine the level • Policy • Program • Clinical • Clarify the challenge 1. Determining ‘best’ practice 2. Getting support for ‘best’ practice 3. Implementing ‘best’ practice

17. DETERMINING ‘BEST’ PRACTICE • Assess the research • Literature analysis • Systematic reviews weighted • The trap of “decision-based evidence making” • Evaluate for your setting • Transferability, context, resources, values • Identify gaps • Identify local evidence, experience of other programs (e.g. QI, evaluation activities)

18. GETTING SUPPORT FOR ‘BEST’ PRACTICE • Identify champions • Align with strategic priorities • Use existing activities and structures • Build collaborative relationships • Integrate research with local evidence • Speak to audience/stakeholder concerns • Use effective communication strategies

19. IMPLEMENTING THE EVIDENCE • Recognize implementation challenge • Plan, accountability, resources • “the personality factor” • “change management” • Communicate evidence effectively • Link to personal, professional objectives • Build collaborative relationships • Respect expertise of stakeholders • Implementation & formative evaluation • Positive forum for identifying barriers

20. SUMMARY • Evidence informed decision-making is important at all levels of health care provision: clinical, planning and policy • Even with limited resources, we have opportunities to address internal barriers to evidence-informed decision-making • Evidence should be used at all three stages of decision-making: determining a solution, getting support for a proposed solution, and implementing the decision.