If We Want More Evidence-based Practice, We Need More Practice-Based Evidence

1. The Shifting Context for Theory and Evaluation in Translational Research If We Want More Evidence-based Practice, We Need More Practice-Based Evidence UCSF Translation-2 Course, April 15, 2008 Lawrence W. Green University of California at San Francisco

2. NIH Roadmap Initiative--translating discoveries into health* The roadmap less traveled?** “The Roadmap identifies the most compelling opportunities in three arenas: new pathways to discovery, research teams of the future, and reengineering the clinical research enterprise” (Zerhouni, p. 63).* **Green LW. Am J Prev Med., 2007; 33(2):137-38, after K. Grumbach. *Zerhouni E. Science 2003, Oct 3;302(5642):63-72 .

3. The Transdisciplinary Blending of Theoretical Traditions

4. Phase 5 Administrative & policy assessment Phase 4 Educational & ecological assessment Phase 2 Epidemiological assessment Phase 1 Social assessment Phase 3 Behavioral & environmental assessment Predisposing Health Promotion Health education Behavior Reinforcing Quality of life Health Policy regulation organization Environment Enabling The Precede-Proceed model as it appeared in the previous two editions of the book* PRECEDE PROCEED Phase 6 Implementation Phase 7 Process evaluation Phase 8 Impact evaluation Phase 9 Outcome evaluation Input Process Short-term social impact Output Short-term impact Longer-term health outcome Long-term social impact *Green & Kreuter, Health Promotion Planning, 2nd & 3rd eds., Mayfield, 1991, 1999.

5. Predisposing, Reinforcing, & Enabling Constructs in Educational/Ecological Diagnosis & Evaluation Policy, Regulatory & Organizational Constructs in Educational & Environmental Development PRECEDE-PROCEED Model Green & Kreuter, Health Program Planning, 4th ed., NY, London: McGraw-Hill, 2005.

6. Evolution of the Model, Bangladesh • Correlates of family planning in Pakistan, 1960s • Green LW., Krotki KJ. Proximity and other geographical factors in the utilization of family planning clinics in Pakistan. Pakistan Dev. Rev. 6:80‑104, 1966. • Green LW. Validity in family planning surveys: Disavowed knowledge and use of contraceptives in a panel study in Dacca, East Pakistan. Public Opin. Q. 32:504, 1968. • Green LW., Krotki KJ. Class and parity biases in family planning programs: The case of Karachi. Soc. Biol.15:235‑251, 1968. • Green LW. East Pakistan: Knowledge and use of contraceptives. Stud. Fam. Planning 1:9‑14, 1969. • Cluster trial of interventions in Dhaka • Green, L.W. Identifying and overcoming barriers to the diffusion of knowledge about family planning. Adv. Fertil. Control 5:21‑29, 1970. • Green, L.W., et al. Field experiment comparing family planning education programs directed at males and females. Int. J. Health Educ. 16:242‑259, 1973.

7. Evolution: The Hopkins Trials • Asthma: the diffusion concept of homophily • Green LW. Toward cost‑benefit evaluations of health education: Some concepts, methods and examples. Health Educ. Monogr. 2 (supp.2):34-64, 1974. • Maiman L, Green LW, Gibson G, Mackenzie EJ. Education for self-treatment by adult asthmatics. J. Am. Med. Assoc. 241:1919-1922, 1979. • Hypertension: the concept of comprehensiveness • Green LW., Levine DM, Deeds SG. Clinical trials of health education for hypertensive outpatients: Design and baseline data. Prev. Med. 4:417‑425, 1975. • Green, L.W., et al. Development of randomized patient education experiments with urban poor hypertensives. Patient Couns. Health Educ. 1:106‑111, 1979. • Levine DM, Green LW, Deeds SG, et al. Health education for hypertension patients. J. Am. Med. Assoc. 241:1700-1703, 1979. • Morisky DE, Levine DM, Green LW, et al. Five‑yearblood‑pressure control and mortality following health education for hypertensive patients. Am J Public Health 73:153‑162, 1983.

8. Phase 3 Educational & ecological assessment Phase 1 Social assessment Predisposing Reinforcing Quality of life Health Enabling The generic representation of the new version of PRECED-PROCEED, with new elements highlighted.* Phase 2 Epidemiological Assessment Phase 4a Intervention Alignment Health Program Genetics Phase 4b Educational strategies Administrative & policy assessment Behavior Policy regulation organization Environment Phase 7 Impact & Outcome evaluation Phase 5 Implementation Phase 6 Process evaluation Green & Kreuter, Health Program Planning, 4th ed., NY, London: McGraw-Hill, 2005.

9. Hallmark 1: Both Procedural and Logical Community engaged? Yes Collect/Analyze data, get consensus, set priorities (Chap 2, B ) 3 NO Social goals & objectives Select & apply procedures for community participation (Ch 2, A) PRECEDE Are best practices & resources for program available, & policies in place? Are behavioral & environmental causes, objectives clear? Are Predisposing, Enabling, and Reinforcing factors clear? Yes Yes Yes Are health objectives clear? Yes If Not If Not If Not Go to Chap 3 If Not Go to Chap 4 Go to part 2 of Chap 3 Go to Chap 5 Implementation & Evaluation Plan PROCEED

10. Where Have All the Data Gone? Longtime Passing… 17 yrs “It takes 17 years to turn 14 per cent of original research to the benefit of patient care” * Original research Submission Unknown 0.5 year 0.6 year 0.3 year 6. 0 - 13.0 years 9.3 years Poyer, 1982 Antman, 1992 Kumar, 1992 Kumar, 1992 Acceptance Publication Bibliographic databases Reviews, guidelines, textbooks Implementation 35% 50% Negative results 18% 46% Lack of numbers, Design issues Lack of numbers, Design issues *Balas, 1995 Inconsistent indexing Poynard, 1985 Dickersin, 1987 Koren, 1989

11. R The Pipeline Fallacy of Producing & Vetting Research to Get Evidence-Based Practice* The 17-year odyssey Practice - Guidelines for Evidence-Based Practice Funding; patient needs, demands; local practice circumstances; professional discretion; credibility & fit of the evidence. Research Synthesis Publication Priorities & Peer Review Peer Review Of Grants Priorities for Research Funding Evidence-based Medicine movement Academic appointments, promotion, & tenure criteria Blame the practitioner or blame dissemination *Based on Green, L.W. From research to “best practices” in other settings and populations. Am J Health Behavior 25:165-178, April-May 2001. Full text: www.ajhb.org/25-3.htm

12. The research indicates that we really should do something with all this research. Diffusion Adoption Quality EBP “Bridging the gap”

13. 5 Ways of Making Research More Relevant for Practice • Making research more theory-based • Setting research & evaluation priorities • Making research findings actionable, usable, relevant (to whom?) • Disseminating & translating (adapting) research to local circumstances, cultures, and personnel • Making evidence more practice-based

14. Filling the Chasm, as Conceived by the U.S. Translation Agency* Practice is We want it to here be here Innovation Implementation Reminiscent of the “Fallacy of the Empty Vessel” from early health education Adoption Education Diffusion TRIP *Carolyn Clancy. Agency for Healthcare Research & Quality 2003.

15. Prototype of Causal Models and Intervention Models Problem Theory: Causes (X)>->->->->->Effects Action Theory Causal Theory: Intervention Models: OUTPUTS (behavioral change, health, quality of life, development) INPUTS (educational, organizational economic, etc.) X ? Different models interpret the content of “X?” according to different theories (or assumptions) about causation and control (mediating variables). Green & Kreuter, Health Program Planning, 4th ed., NY, London: McGraw-Hill, 2005.

16. Examples of Causal Theories on Which PRECEDE-PROCEED Model is Based • Psychological theories: X includes a behavior, and its antecedents such as attitudes, beliefs, values, perceptions, and other cognitive variables • Sociological theories: X includes social norms, networks, diffusion, organizational functioning, and inter-organizational exchange & coalitions. • Economic theories: X includes consumer behavior and organizational response to consumer demand; governmental subsidies or incentives, taxes. • Pathophysiological theories: X includes organisms or environmental exposure processes.

17. Action Theory and Program Theory Use Causal Theories to Link Intervention and Outcomes Intervention(s) or Program Outcome Variable(s) Cause(s) E.g., Fear-arousal communication Belief in susceptibility Change portion size CausalTheory Action Theory Health Belief Model Fear-arousal theory Program Theory Protection-Motivation Communication *Adapted from Suchman, 1967, pp. 84, 173; Weiss, 1970; Chen, 1990, p. 250; Donaldson, 2001, pp. 473-487; Green & Kreuter, 2005, p. 200.

18. Mediating (“X”) and Moderating (Contextual) Variables Mediator Outcome Fear Intervention Variable(s) or Program Portion size Communication Mediator Belief in Severity Moderators Moderators Age, SES Gender, culture Green & Kreuter, Health Program Planning, 4th ed., NY: McGraw-Hill, 2005, p. 204.

19. Population change defined by diffusion theory early diffusion stages: predisposing factors middle diffusion stages: reinforcing factors late stages of diffusion: enabling factors Individual change defined by psycho-logical theory precontemplation to contemplation: predisposing factors preparation & action stages: enabling factors maintenance: rein-forcing factors Level and Stage of Change as a First Consideration

21. Tobacco Vending Machine Ordinances Number of Ordinances (Cumulative) 180 Total Ban Partial Ban 160 140 120 100 80 60 40 20 0 1985 1986 1987 1988 1989 1990 1991 1992* Year * Through September 1992. Source: National Institutes of Health, National Cancer Institute (1993). Smoking and Tobacco Control - Monograph 3. Major Local Tobacco Control Ordinates in the U.S. US Dept. of Health and Human Service. Public Health Service, National Institutes of Health. NIH Publ. No. 93-3532.

22. Glasgow’s RE-AIM “Law of halves” e.g., ULTIMATE IMPACT OF MAGIC DIET PILL Dissemination StepConcept% Impacted 50% of Clinics Use Adoption 50% 50% of Clinicians Prescribe Adoption 25% 50% of Patients Accept Medication Reach 12.5% 50% Follow Regimen Correctly Implementation 6.2% 50% of Those Taking Correctly Benefit Effectiveness 3.2% 50% Continue to Benefit After 6 Months Maintenance 1.6%

23. Behavior Health Environment Evaluation tasks begin at Phase 1, and continue through as many diagnostic, implementation, and follow-up evaluation phases as required. Phase 2 Epidemiological, Behavioral and Environmental Assessment Phase 3 Educational & ecological assessment Phase 1 Social assessment Phase 4a Intervention Alignment Precede Evaluation tasks: Specifying measurable objectives and baselines. Predisposing Health Program Genetics Phase 4b Administrative & Policy Assessment Educational strategies Reinforcing Quality of life Policy regulation organization Enabling Proceed Evaluation Tasks: Monitoring & Continuous Quality Improvement Short-term impact Longer-term health outcome Short-term social impact Long-term social impact Input Process Output Phase 5 Implementation Phase 6 Process evaluation Phase 7 Impact and outcome evaluation

24. Theory Implied in Phases 3-4 of PRECEDE* Phase 4: Intervention Alignment, Administrative And Policy Assessment Phase 3: Educational and Ecological Assessment Predisposing Knowledge, Attitudes Beliefs Cultural Values Perceptions 1 Genetics and Human Biology Matching and Mapping Interventions with Evidence & Theory 5 2 4 Direct Communications 17 3 Reinforcing Health prof’ls, parents, teachers, employers, peers, vendors etc. Health Education, Mass Media, Advocacy, Training Behavior and Lifestyle 6 7 Indirect Communi- cations 12 8 16 13 11 10 Enabling Availability of resources Accessibility Skills Ecosystem 14 9 Policy, Regulation, Organization 15 Environment *Green & Kreuter, 2005, p. 149.

25. Summary of the Narrowing Phasing of PRECEDE-PROCEED Phase 1. Social & Quality-of-Life Assessments & Situation Analysis Phases 2-3. Epidemi- ological, Educational & Ecological Assessments Phase 4. Administrative & Policy Assessment, PROCEED to Action, Formative evaluation For each goal, assess causes, determinants Objec- tives Assess theory, evi- dence for change Select methods, Assign roles Asses importance, feasibility of each Setting Priorities For each objective, assess resources, polices Vision Goals Set priorities Felt needs, Assets, Concerns, Aspirations Tactics Strategy Phase 5 Implementation Pretest Methods Activate Timelines for Training, Interventions Phase 7 Phase 8 Phase 6 Evaluation…of methods…intermediate objectives…ultimate goals Green & Kreuter, Health Program Planning, 4th ed., NY: McGraw-Hill, 2005, p. 65.

26. RFA (PAR) from NIH, 2006 • Applications to “ identify, develop, and refine effective and efficient methods, structures, and strategies that test models to disseminate and implement research-tested health behavior change interventions and evidence-based prevention, early detection, diagnostic, treatment, and quality of life improvement services into public health and clinical practice settings.” • Two problems with these framings of the issue: • Are the “research-tested interventions” adequate? • Are they appropriate to other settings, populations? To illustrate the first problem:

27. Canadian Cancer Society RFP for a Review to Answer 4 Questions Are group counseling programs for smoking cessation effective? If so, what is the optimal content of the sessions? What is the optimum number and frequency of sessions that should be offered? What are the characteristics of the most effective facilitators?

28. University of Waterloo Results* • A comprehensive literature review of over 40 years of published and unpublished studies • Deficiencies in purpose, design and reporting • Research could answer only the first of 4 questions: that group programs for smoking cessation are effective. *Manske SR, Miller S, Moyer C, Phaneuf MR, Cameron RC. Best practice in group- based smoking cessation: Results of a literature review. AJHP 18:409-23, 2004.

29. Evidence-Based Medicine and Patient-Centered Medicine* Information of importance to patient choice that is not even potentially of “evidence-based type.” Area where there is currently good evidence-based information of importance to patients in making choices. C A A “Good evidence” B Potential for “good evidence” C Information of potential importance to patients in making health care choices Information of importance to patient choice that is potentially of evidence- based type. B *In A.L. Cochrane, from T.Hope. Evidence-based patient choice and the doctor-patient relationship. In But Will It Work, Doctor? Kings Fund, London, 1997, 20-24.

30. Evidence-Based Public Health Information of importance to community choice that is not even potentially of “evidence-based type.” Area where there is currently “good evidence-based” information of importance to communities in making choices. C A A “Good evidence” B Potential for “good evidence” C Information of potential importance to communities in making health choices B Information of importance to community choice that is potentially of “evidence- based” type.

31. Issues for Evidence-Based Practice and Translating Research to Practice • Making practice more theory-based • Setting research priorities • Making research findings actionable, usable, relevant within settings • Translating research from outsideto local circumstances, cultures, personnel • Making evidence more practice-based

32. Priority-Setting for Health Research* Population Level • Program • Evaluation CDC • Community & Statewide • Effectiveness Trials • Surveillance PBRNs, CQI Applied Research & Development Clinical Trials Basic Research Demonstration & Education Research Clinical Investigations NIH Molecular Level T2 T1 Knowledge Acquisition Knowledge Transfer Knowledge Translation Knowledge Validation *Green LW, Popovic T, et al. CDC Futures Workgroup on Research. Atlanta, 2004.

33. The Internal Validity Drift of Health Sciences Evidence “Lost in Translation” • Evidence-based medicine movement taken to scale in general practice & health promotion • The peer review preferences for experimental control and certainty of causation • The publishing preferences for RCTs and positive results • The limitations of print space driving out richer description of interventions, protocols, procedural lessons, subgroup variations • But a more “natural” type of practice-based evidence has greater influence on multi-level program planning, practice & policy…

34. Change in Per Capita Cigarette ConsumptionCalifornia & Massachusetts vs Other 48 States, 1984-1996 5 0 -5 Percent Reduction -10 -15 -20 -25 Other 48 States California Massachusetts 1984-1988 1990-1992 1992-1996

35. Issues for Evidence-Based Practice and Translating Research to Practice • Making practice more theory-based • Setting research priorities • Participatory research to make findings actionable, usable, relevant within settings • Translating research from outsideto local circumstances, cultures, personnel • Making evidence more practice-based

36. Some Benefits of Participatory Research in Practice-Based Evidence • Results are relevant to interests, circumstances, and needs of those who would apply them • Results are more immediately actionable in local situations for people and/or practitioners • Generalizable findings more credible to people, practitioners and policy makers elsewhere because they were generated in partnership with people like themselves • Helps to reframe issues from health behavior of individuals to encompass system and structural issues. Green LW, Mercer SL. Am J Public Health Dec. 2001.

37. Definition and Standards of Participatory Research for Health* Systematic investigation… Actively involving people in a co-learning process… For the purpose of action conducive to health** --not just involving people more intensively as subjects of research or evaluation *Green, George, Daniel, et al., Participatory Research…Ottawa: Royal Society of Canada, 1997. www.lgreen.net/guidelines.html

38. The Lenses of Scientists, Health Professionals and Lay People Subjective Indicators of Health Professional, Scientific Layperson Objective Indicators of Health

39. Issues for Evidence-Based Practice and Translating Research to Practice • Making practice more theory-based • Setting research priorities • Making research findings actionable, usable, relevant: participatory research • Translating research to local cultures & circumstances: External validity & “fidelity” vs adaptation • Making evidence more practice-based

40. Building Policy and Practice from Evidence + Theory • Not starting with theory and looking for problems on which to test them, but starting with problems and looking for theories to help us solve them* • Evidence on solutions generalizes to other circumstances, settings, & populations in the form of either replication or theory • Replication is limited by the infinite number of context-population combinations • "In theory, theory and practice are the same thing. In practice they're not..“ -Jan L.A. van de Snepscheut • “All models are wrong. Some are useful” --Box *Green LW. Public health asks of systems science… Amer J Public Health 96, March 2006.

41. “Fidelity” vs Adaptation* • Researchers test an intervention for its efficacy • Rigorous test (efficacy) qualifies it for official lists of “evidence-based practices” and guidelines • Practitioners try to incorporate it into their programs in other populations, circumstances • Poor fit produces failure of program • Practitioners are blamed for not implementing with “fidelity” • Now buy the producers’ training program * Green LW, Glasgow RE, …external validity…Evaluation & the Health Professions, Mar. 2006.

42. Efficacy vs. Effectiveness: • Efficacy. The tested impact of an intervention under highly controlled circumstances. • Effectiveness. The tested impact of an intervention under more normal circumstances (relativelyless controlled, real-time, “typical” setting, population, and conditions). • Broad Program Evaluation. The tested impact of a blended set of interventions on larger systems and populations. “Natural Experiments” with minimal control, maximum variability.

43. The Trade-offs • Efficacy. Maximizes internal validity, i.e., the degree to which one can conclude with confidence that the intervention caused the result. • Effectiveness. Maximizes external validity,* i.e., the degree to which one can generalize from the test to other times, places, or populations. • Program Evaluation. Maximizes reality testing in particular settings, & with the combination of interventions at multiple levels required for public health effect. * Green LW, Glasgow RE, …external validity…Evaluation & the Health Professions, Mar. 2006.

44. Issues for Evidence-Based Practice and Translating Research to Practice • Blending evidence-based practice with theory-based practice • Setting research priorities • Making research findings actionable, usable, relevant: Participatory Research • Translating research to local circumstances • Making evidence more practice-based: the centrality of evaluation and continuous quality improvement research

45. Mediating and Moderating Variables Mediator Intervention Outcome or Program Variable(s) Mediator Moderators Moderators Green & Kreuter, Health Program Planning: An Educational and Ecological Approach. 4th ed. New York: McGraw-Hill, 2005. Green & Glasgow, E&HP, 2006.

46. Aligning Evidence with (and deriving it from) Practice: Matching, Mapping, Pooling and Patching • Matching ecological levels of a system or community with evidence of efficacy for interventions at those levels • Mapping theory to the causal chain to fill gaps in the evidence for effectiveness of interventions • Pooling experience to blend interventions to fill gaps in evidence for the effectiveness of programs in similar situations • Patching pooled interventions with indigenous wisdom and professional judgment about plausible interventions to fill gaps in the program for the specific population *Green & Kreuter, Health Program Planning: An Educational and Ecological Approach. 4th ed. NY: McGraw-Hill, 2005, Chapter 5. Green & Glasgow, 2006.

47. 3 Conceptualizations of the Gap Between Research & Practice • Practitioners need to receive the lessons of research and put them into practice. • Research and practice are entirely separate disciplines and each must develop their own answers to their own problems • Research and practice have complementary perspectives and skills that need to be used together to address the real need, collaborative knowledge production. • Add to this the need to include the patient’s perspective. Whose perspective prevails? Van De Ven A, Johnson P. Knowledge for theory and practice. Academy of Management Review. 2006;31(4).

48. The Bridge (not the Pipeline) from Research to Practice and Back • If we want more evidence-based practice, we need more practice-based evidence. • The importance of practitioners and policy-makers in shaping the research questions. • Practitioners and their organizations represent the structural links (and barriers) to addressing the important determinants of health behavior at each level. Engage them, not at passive recipients, but as partners… *Green, L.W. From research to “best practices” in other settings and populations. Am J Health Behavior 25:165-178, April-May 2001. Full text: www.ajhb.org/25-3.htm.

49. The Vision for Translation 2 A future in which we would not need to ask how to get more evidence-based practice, rather How to sustain the engagement of students, practitioners, patients and communities in a participatory process of practice-based research and program evaluation? How to adapt the “best practices” guidelines through best processes of collecting data to diagnose the biopsychosocial needs of their patients and communities…

50. Translation 2 Vision (expanded) How to match the proposed evidence-based interventions to those needs, filling gaps in the evidence-based interventions with the use of theory and mutual consultation, and prospective testing of complementary interventions The cumulative, building-block tradition of evidence-based medicine from RCTs would be complemented by a parallel strengthening and support of a tradition of participatory research and evaluation conducted in practice settings.