Advancing the science of knowledge translation

1. Advancing the science of knowledge translation Jeremy Grimshaw for KT Canada Clinical Epidemiology Program, OHRI Department of Medicine, University of Ottawa Canada Research Chair in Health Knowledge Transfer and Uptake

2. Background ‘All breakthrough, no follow through’ Woolf (2006) Washington Post op ed • Much of the US $100 billion/year worldwide investment in biomedical and health research is wasted because of dissemination and implementation failures

3. Background Institute of Medicine;Clinical Research Roundtable, Sung et al. JAMA 289:1278,2003

4. Background • Consistent evidence of failure to translate research findings into clinical practice • 30-40% patients do not get treatments of proven effectiveness • 20–25% patients get care that is not needed or potentially harmful • This has led to increased policy and research interest into efforts to bridge the evidence-practice gap to improve quality of care Schuster, McGlynn, Brook (1998). Milbank Memorial Quarterly Grol R (2001). Med Care Seddon (2001) QHC

5. Knowledge translation CIHR definition • Knowledge translation is a dynamic and iterative process that includes the synthesis, dissemination, exchange and ethically-sound application of knowledge to improve the health of Canadians, provide more effective health services and products and strengthen the healthcare system.

6. Knowledge translation

7. applied health research capacity building co-optation - cooperation - competing diffusion* dissemination* getting knowledge into practice impact Implementation* knowledge communication knowledge cycle knowledge exchange knowledge management knowledge translation knowledge mobilization knowledge transfer linkage and exchange popularization of research, research into practice research mediation research transfer research translation science communication teaching “third mission” translational research transmission utilization *cited most frequently KT terms encountered

8. Knowledge translation • Knowledge translation is about ensuring that: • stakeholders are aware of and use research evidence to inform their decision making • (research is informed by current available evidence and the experiences and information needs of stakeholders)

9. Knowledge translation research • Knowledge translation is a human enterprise that can be studied to understand and improve knowledge translation approaches • Knowledge translation research is the scientific study of the determinants, processes and outcomes of knowledge translation. • Goal is to develop a generalisable empirical and theoretical basis to optimise KT activities

10. Knowledge translation research • Knowledge translation research relatively new field in health research • Inherently interdisciplinary • Wide range of disciplines need to be engaged • Clinical • Health services research • Behavioural and organisational • Design and engineering • Methodologists • Broad range of forms of enquiry needed

11. Knowledge translation research • Knowledge synthesis (to identify the knowledge for KT); • Research into the evolution of and critical discourse around research evidence; • Research into knowledge retrieval, evaluation and knowledge management infrastructure; • Identification of knowledge to action gaps; • Development of methods to assess barriers and facilitators to KT; • Development of the methods for optimizing KT strategies; • Evaluations of the effectiveness and efficiency of KT strategies; • Development of KT theory; and • Development of KT research methods.

12. Knowledge translation research • Knowledge synthesis (to identify the knowledge for KT); • Research into the evolution of and critical discourse around research evidence; • Research into knowledge retrieval, evaluation and knowledge management infrastructure; • Identification of knowledge to action gaps; • Development of methods to assess barriers and facilitators to KT; • Development of the methods for optimizing KT strategies; • Evaluations of the effectiveness and efficiency of KT strategies; • Development of KT theory; and • Development of KT research methods.

13. Developing methods to assess barriers and facilitators to KT • Structural (e.g. financial disincentives) • Organisational (e.g. inappropriate skill mix, lack of facilities or equipment) • Peer group (e.g. local standards of care not in line with desired practice) • Individual (e.g. knowledge, attitudes, skills) • Professional - patient interaction (e.g. problems with information processing)

14. Developing methods to assess barriers and facilitators to KT • Formal assessment of context, likely barriers to KT • Mixed methods • Literature review • Informal consultation • Focus groups • Surveys • Needs interdisciplinary perspective

15. Developing methods to assess barriers and facilitators to KT

16. Developing methods to assess barriers and facilitators to KT • Knowledge • Skills • Professional role and identity • Beliefs about capabilities • Beliefs about consequences • Motivation and goals • Memory, attention and decision processes • Environmental context and resources • Social influences • Emotion • Behavioural regulation • Nature of the behaviour

17. Developing methods to assess barriers and facilitators to KT

18. Developing methods to assess barriers and facilitators to KT • Focus groups, theoretical approach • Ongoing cluster RCT to develop and evaluate intervention to improve GP management of low back pain ( diagnostic imaging,  exercise) • Conducted focus group with 42 general practitioners • Focus group analysis based upon the BPS domains

19. Developing methods to assess barriers and facilitators to KT ISLAGIATT principle ‘It Seemed Like A Good Idea At The Time’ Martin P Eccles

20. Developing methods for optimizing KT strategies • Choice of dissemination and implementation should be based upon: • ‘Diagnostic’ assessment of barriers • Understanding of mechanism of action of interventions • Empirical evidence about effects of interventions • Available resources • Practicalities, logistics etc

21. Developing methods for optimizing KT strategies • Intervention mapping • Specify intervention objectives • Select methods and strategies • Design program • Usability studies

22. Developing methods for optimizing KT strategies • We have found it useful to distinguish: • What we are trying to change • Why are we trying to change it? (constructs: barriers and enablers) • How are we going to change it, including • Behaviour change technique • Context: the mode of delivery (eg group meeting, DVD) • Content: how the technique will be operationalised

23. Developing methods for optimizing KT strategies

24. Matching behaviour change techniques to theoretical constructs agree use; agree don’t use; disagreement; indefinite

25. Developing methods for optimizing KT strategies – IMPLEMENT example • What we are trying to change? • Knowledge of what red flags are and skills in how to identify them and diagnose acute low back pain • Why are we trying to change it? • Construct: Knowledge (GP) • How are we going to change it? • Technique: Information provision • Context: educational meeting; advertising campaign; DVD • Content: Behavioural task with feedback; eg in pairs run through the process; quiz?; practise use of an algorithm

26. Developing methods for optimizing KT strategies – IMPLEMENT example • What we are trying to change? • Skills and beliefs about capabilitiesrelated to giving advice to stay active (inc what advice to give) • Why are we trying to change it? • Construct: Skills, Knowledge (GP), Beliefs about capabilities • How are we going to change it? • Technique: behavioural rehearsal; role play; scripting • Context: educational meeting; advertising campaign; DVD • Content: Participants write down wording of their last or usual message to stay active and then discuss in groups of 2-4 and create a script they feel comfortable with. Then role play with feedback. Educators model if necessary. Idea is that GPs should feel comfortable with wording of their own script, compared with a generic script, so that it is in their own language and consistent with the way they speak, behave, etc

27. Developing methods for optimizing KT strategies – IMPLEMENT example • Two small group educational meetings • Homework • DVD, educational materials • Patient leaflets

28. Session 1

29. Developing methods for optimizing KT strategies – Usability studies • Develop prototype intervention • Test prototype in 5 to 8 subjects to review content and format using ‘think aloud’ methodology. These sessions will be audio recorded and the results transcribed and analysed. • In general a modest number of subjects are required for usability testing (e.g. 8-9 subjects), and often 4 to 5 are necessary to identify 80% of the usability problems. • Cycles of design, development and testing will be completed until no further major revisions are needed.

30. Evaluating the effectiveness and efficiency of KT strategies • Causal description – did our KT strategy lead to improve knowledge use • Causal explanation – why did our KT strategy work/not work (understanding of mediating pathways) • Economic evaluation • Understanding of potential effect modifiers (context, targeted group, targeted behaviour, variations in intervention) • Many current KT evaluations fail to address some or all of these issues

31. Evaluating the effectiveness and efficiency of KT strategies • Pragmatic largely cluster randomised trials are optimal design for establishing causal description. • Design aspects can be used to enhance informativeness of RCTs • Multiple arm trials, factorial designs • Policy friendly designs • Step wedge designs, balanced incomplete block designs • However for logistical, pragmatic and ethical reasons, quasi experimental designs may be needed.

32. Evaluating the effectiveness and efficiency of KT strategies

33. Evaluating the effectiveness and efficiency of KT strategies • Pragmatic 2 x 2 factorial design of two forms of educational materials – replicated for three behaviours (aggressive cardiovascular risk management in diabetes, diabetic eye screening and use of thiazides for first line hypertension management) • Largest implementation trial to date – approx 6,500 family practices in Ontario • No statistically significant differences

34. Evaluating the effectiveness and efficiency of KT strategies • Other forms of enquiry needed to determine causal explanation • Intervention fidelity studies • Process evaluations (qualitative case studies) • Theory based process evaluations (mediating pathways) • Secondary analyses (moderator analyses) • Temporal analyses • Embedded economic evaluations

35. Evaluating the effectiveness and efficiency of KT strategies

36. Evaluating the effectiveness and efficiency of KT strategies • Conducted a theory based process evaluation alongside OPEM trial • Hypothesised that OPEM interventions would likely be mediated through changes in intentions, attitudes and social norms • Administered TPB survey before and after intervention for two replications (thiazides, diabetic screening) • Before data demonstrated very positive intentions, attitudes and social norms – possibly suggesting ‘psychological ceiling effect’, family doctors able to identify lots of post intention barriers to behaviours • Theory based process evaluation aided interpretation of the study results

37. Evaluating the effectiveness and efficiency of KT strategies Need to build a cumulative science • O’Brien (2007) SR of 66 RCTs of Academic detailing - Median effect across 16 RCTs of prescribing behaviour 4.8% absolute improvement, interquartile range 3.0% to 6.5% • WE DO NOT NEED FURTHER TWO ARM TRIALS OF ACADEMIC DETAILING VS CONTROL AS THEY WILL LIKELY PROVIDE LITTLE NEW INFORMATION • Need for increased use of multi arm trials/factorial trials to increase informativeness of trials

38. Evaluating the effectiveness and efficiency of KT strategies Need to build a cumulative science

39. Developing KT theory More theory, less theories needed • Multiple theories and frameworks of individual and organizational behavior change. • Most professional behavior change frameworks are descriptive and normative rather than predictive. • Few have been operationalised in detail • Many have not been prospectively evaluated. • Few head-to-head comparisons of different theories • Need for predictive theories that incrementally improve likelihood of successful implementation. • Need for rigorous evaluation of candidate theories

40. Developing KT theory

41. Developing KT methods • Substantial methodological differences between cluster randomized trials and conventional randomized trials pose serious challenges to the current conceptual framework for research ethics

42. Developing the field of KT research • KT is a relatively new field - few health researchers have been engaged in the field for more than 10 years • Substantive level of research activity • Cochrane Effective Practice and Organisation of Care (EPOC) group register includes over 6,000 RCTs and quasi experiments of interventions to improve health care delivery and health care services • Increasing funding and reporting opportunities for knowledge translation research • Move towards research programs and laboratories

43. Developing the field of KT research www.implementation science.com

44. Developing the field of KT research Challenges • Advocating for ongoing research support • Incremental development of generalisable knowledge (requires greater standardisation of concepts, terminology, methods and reporting) • Facilitating interdisciplinarity • Capacity development • Career progression • ?Establish a formal field • Translating Knowledge translation research

45. Contact details • Jeremy Grimshaw jgrimshaw@ohri.ca • EPOC epoc@uottawa.ca http://www.epoc.uottawa.ca/index.htm • Implementation Science http://www.implementationscience.com