Onil Bhattacharyya, MD, PhD and Monika Kastner, PhD

1. KNOWLEDGE TRANSLATION CANADA Making Guidelines Easier to Follow: Bridging Best Evidence and a Clear Message Onil Bhattacharyya, MD, PhD and Monika Kastner, PhD Li Ka Shing Knowledge Institute of St. Michael’s Hospital University of Toronto March 11, 2010

2. Collaborators • Elizabeth Estey • Laure Perrier • Sharon Straus • Merrick Zwarenstein • Jeremy Grimshaw • Ian Graham • Canadian Diabetes Association (CDA) • Canadian Thoracic Society (CTS) • C-CHANGE Funded by KT Canada Clinical Research Initiative (CFI/CIHR)

3. Presentation Outline • Project objectives • Background • Current guideline tools • Study framework and design

4. Background “Guidelines are often too narrowly focused on single diseases and are not patient focused. Patients seldom have single diseases and few if any guidelines help physicians manage complexity. Paradoxically, guidelines are often too comprehensive, covering every possible intervention that could be possible for a patient with that single disease….If there is a main message in such guidelines, it is likely to be lost in the minutiae”1 1Shaneyfelt TM, et al. JAMA 2009; 301(8):868-869.

5. Background- Guidelines on the frontlines “I’m stumbling because I haven’t gotten to the point where I’ve assigned greater or lesser importance to one or the other, like there’s a number of them and I try to do them all, but if I had to choose one, I’m not sure what to choose, maybe an aspirin (laugh)… I don’t know what criteria I would use to, right now, to prioritize them.”2 2 SLZ study participant (MD1).

6. Guideline Implementation • Extrinsic strategies • Changing environment/providers • Highly variable costs and impact • Intrinsic strategies • Changing guidelines themselves • Implementability: a set of perceived characteristics of guidelines that predict the relative ease of their implementation in practice3 • Potentially highly cost-effective 3Shiffman RN, et al. BMC Med Inform Decis Mak 2005; 5:23.

7. Assessment of Existing Tools Existing guideline tools can… • Assess the methodological quality of guidelines (AGREE)4 • Inform developers about implementation issues (GLIA)5 • Adapting existing guidelines to other settings (ADAPTE)6 But do not completely… • Consider how content, format and structure could be modified to increase uptake • Operationalize the processes that could enhance guideline implementability • Identify and resolve the differing views of guideline developers and end users (i.e. providers) 4AGREE Collaboration; Qual Saf Health Care 2003;12:18-23 5Shiffman RN, et al. BMC Med Inform Decis Mak 2005:5:23; 6Fervers B, et al. In J Qual Health Care 2006;18:167-76.

8. Project Objectives • To determine whether changing the attributes of clinical practice guidelines (CPGs) will improve implementation • Do literature synthesis on guideline characteristics and uptake • Incorporate perceptions of guideline developers and end users • Develop and evaluate a new guideline implementability tool (GUIDE-IT)

9. Project Objectives • To determine whether changing the attributes of clinical practice guidelines (CPGs) will improve implementation • Do literature synthesis on guideline characteristics and uptake • Incorporate perceptions of guideline developers and end users • Develop and evaluate a new guideline implementability tool (GUIDE-IT)

10. Guiding Frameworks • Graham’s “Knowledge-to-Action” (KTA) framework • Medical Research Council (MRC) framework for complex interventions

11. Study Design Overview 1: Literature review (KTA Steps 1-2; MRC Phase 1) 2: Tool development process (KTA steps 3-4; MRC Phase 2) 3: Process Evaluation (KTA steps 5-7; MRC Phase 3) Evaluate impact in RCT Use tool to revise guideline Monitor use of tool Problem: Guidelines hard to implement Implementability knowledge synthesis Sustain use of tool Test in focus groups Build prototype Test usability Design conceptual tool

12. Literature review 1: Literature review (KTA Steps 1-2; MRC Phase 1) 2: Tool development process (KTA steps 3-4; MRC Phase 2) 3: Process Evaluation (KTA steps 5-7; MRC Phase 3) Evaluate impact in RCT Use tool to revise guideline Monitor use of tool Problem: Guidelines hard to implement Implementability knowledge synthesis Sustain use of tool Test in focus groups Build prototype Test usability Design conceptual tool

13. Evidence synthesis • Traditional systematic review may miss literature which is multidisciplinary, multifaceted, and under-theorized • Synthesis draws on different approaches to combine qualitative and quantitative literature • Core question: • What characteristics of guidelines affect uptake?

14. Approaches to Literature Synthesis Barnett-Page and Thomas, BMC Med Res Method, 2009

15. Approaches to Literature Synthesis Barnett-Page and Thomas, BMC Med Res Method, 2009

16. Realist Review • Explicitly theory-driven approach to evidence synthesis • Underlying theory= modifying characteristics of guidelines will increase their uptake Develop a theoretical framework for implementability Pawson RT, et al. J Health Serv Res Policy 2005;10:21-34

17. Meta-narrative Review • Unfolding “storylines” of research in a particular scientific tradition • Body of theoretical knowledge and linked set of primary studies • Analysis across different disciplines Develop a narrative account of implementability Greenhalgh, T, et al. Soc Sci & Med 2005; 61:417-30

18. Search Strategy • Preliminary set of core articles • “Snowball sampling” • Related-articles search • Electronic database search

19. Preliminary Search & Data Collection

20. Preliminary Findings • Series of guideline attributes drawn from core articles: • Actionable, clear, specific, complex, evidence-based, flexible, feasible • Conceptualized as a series of trade-offs • Attributes of recommendations vs. guideline as a whole

21. Preliminary Framework • Complex • Flexible • Evidence-based TRADE-OFFS • Actionable • Clear • Specific FEASIBILITY Guideline Uptake

22. Analysis • Goal: Develop a conceptual model of implementability • Identify and define key dimensions • Integrative vs. interpretive reviews • Draw on multidisciplinary perspectives • Map theories and narratives within and across each discipline

23. Multidisciplinary Perspectives Psychology: Cognitive features of the user Gist, Fuzzy Trace Theory Diffusion of Innovations: Characteristics of the “thing” (ie. CPGs, the innovation) ? Human Factors Engineering: Human systems integration Management/Marketing: Persuasive communication

24. Integrative analysis • Diffusion of innovations • Complexity • Clarity • Evidence-based Communication/ Medicine • Fuzzy Trace Theory • Promote gist vs. verbatim memory Psychology • Major Implementability Attributes • Actionable • Clear • Feasible • Complex • Evidence-based • Persuasive Communication • Informative vs. persuasive communication • Directive recommendations Management/ Marketing • Human systems integration • Learnability • Self-confidence • Usability Human Factors Engineering

25. 2: Tool development process Test in focus groups Build prototype Test usability Design conceptual tool Assess barriers to using the tool Tool Development Process 1: Literature review KTA Problem: Guidelines hard to implement KTA Knowledge: Implementability knowledge synthesis

26. Conceptual tool design • Conceptual design of a guideline implementability tool (GUIDE-IT) • Key guideline attributes from review • Implementability elements of existing tools • Input from content experts

27. Focus Groups • Focus groups with guideline developers and end-users to assess • Perception of guideline development process • What is important in guidelines • What guideline attributes facilitate/hinder uptake

28. FG Study Flow Focus Groups • Focus • Group 1 • Guideline developers • Revise draft guideline • Focus • Group 2 • Guideline users (providers) • Comment on revised guideline • Focus • Group 3 • Mixed group • Resolve conflicts about implementability dimensions • Focus • Group 4 • Mixed group • Feedback on tool design

29. Prototyping GUIDE-IT • Transform the conceptual design of GUIDE-IT into a functioning prototype using: • FG findings • Implementability components of existing tools • Consultation with content experts

30. Usability testing • Providers work through standardized problems with draft guideline • Provider feedback to guideline developers, who use tool to revise recommendations • Tool tested with both groups to identify standard usability problems Kushniruk AW & Patel VL. J Biomed Inform 2004;37(1):56-76.

31. 3: Tool testing Evaluate impact in RCT Use tool to revise guideline Monitor use of tool Sustain use of tool Process Evaluation 1: Literature review 2: Tool development process Problem: Guidelines hard to implement Implementability knowledge synthesis Test in focus groups Build prototype Test usability Design conceptual tool Assess barriers to using the tool

32. Implement, Monitor, Evaluate • Objectives: • To test the use of GUIDE-IT • To monitor its use through a controlled trial

33. Evaluation Framework • Old and new versions of guideline recommendations (modified using GUIDE-IT) presented on a web-based platform • Study comparing impact of original or revised recommendations on clinical decision making • RCT study design based on previous work by Shekelle et al10 10Shekelle PG, et al. Health Services Research 2000;34(7):1429-48.

34. Proposed RCT Study Design Assessed for eligibility N = ? Excluded N = ? Randomized N = ? Group A Vignettes + Standard recommendations N = ? Group B Vignettes + Revised recommendations N = ? Group C (Control) Vignettes only (no recommendations) N = ? Follow-up N = ? Follow-up N = ? Follow-up N = ? Analysis N = ? Analysis N = ? Analysis N = ?

35. Areas for discussion • Realist review & Meta-narrative review: • What do you think of the structure of the review? • Response to initial framework • Assessment of guideline tools: • Are we missing any existing tools?

36. Areas for Discussion • Tool design: • How do you envision GUIDE-IT? • What do you think it should look like? • Overall study design • What are your thoughts on the methods and approach?