PRECEDE-PROCEED: Down the Yellow-Brick Road to Optimal Planning, Implementation and Evaluation

1. PRECEDE-PROCEED: Down the Yellow-Brick Road to Optimal Planning, Implementation and Evaluation Lawrence W. Green University of California at San Francisco and Judith M. Ottoson San Francisco State University NW Chapter of Society for Public Health Education Seattle, May 30, 2013

2. Some Challenges of Planning Programs in Public Health • Health behavior & environmental change are driven by more than knowledge, beliefs, and attitudes (“motivation”) • Health behavior & environmental changes usually must be sustained over long periods to achieve health benefits • Communications must be supported by policies and programs, regulations and organization • Must adapt programs to different populations and settings, with sensitivity to their differences • In summary: the challenges require more than merely importing and imparting information, knowledge, or even wisdom…

3. Predisposing, Reinforcing, & Enabling Constructs in Educational/Ecological Diagnosis & Evaluation Policy, Regulatory & Organizational Constructs in Educational & Environmental Development PRECEDE-PROCEED Model Green & Kreuter, Health Program Planning, 4th ed., NY, London: McGraw-Hill, 2005. For bibliography of >1000 published applications, go to www.lgreen.net

4. Prototype of Causal Models and Intervention Models Problem Theory: Causes (X)>->->->->->Effects Action Theory Causal Theory: Intervention Models: OUTPUTS (behavioral change, health, quality of life, development) INPUTS (educational, organizational economic, etc.) X ? Different models interpret the content of “X?” according to different theories (or assumptions) about causation and control (mediating variables). Green & Kreuter, Health Program Planning, 4th ed., NY, London: McGraw-Hill, 2005.

5. Examples of Causal Theories on Which PRECEDE-PROCEED is Based • Psychological theories: X includes a behavior, and its antecedents such as attitudes, beliefs, values, perceptions, and other cognitive variables • Socio-cultural theories: X includes social norms, networks, diffusion, organizational functioning, and inter-organizational exchange & coalitions. • Economic theories: X includes consumer behavior and organizational response to consumer demand; governmental subsidies or incentives, taxes. • Pathophysiological theories: X includes organisms or environmental exposure processes.

6. Behavior Health Environment Planning & Evaluation Phases Phase 1 Social and Quality of Life Assessment Phase 2 Epidemiological, Behavioral and Environmental Assessment Phase 3 Educational & Ecological assessment Phase 4a Intervention Alignment Precede Evaluation tasks: Specifying measurable objectives and baselines. Predisposing Health Program Genetics Phase 4b Administrative & Policy Assessment Educational strategies Reinforcing Quality of life Policy regulation organization Enabling Proceed Evaluation Tasks: Monitoring & Continuous Quality Improvement Short-term impact Longer-term health outcome Short-term social impact Long-term social impact Input Process Output Phase 5 Implementation Phase 6 Process evaluation Phases 7-8 Impact and outcome evaluation

7. Phases 2-4 of PRECEDE. Phase 4: Intervention Alignment, Administrative And Policy Assessment Phase 3: Educational and Ecological Assessment Phase 2: Epidemio- logical & Behavioral Assessment Predisposing Knowledge, Attitudes Beliefs Cultural Values Perceptions Genetics and Human Biology Matching and Mapping Communications with Evidence, Theory & Policy Direct Communications Reinforcing Influence from parents, teachers, employers, peers, etc. Communications: Health education, Training, Political, Media advocacy Behavior and Lifestyle Indirect Communi- cations Enabling Availability of resources Accessibility Skills, Restrictions Ecosystem Policy, Regulation, Organization Environment

8. Phases 3-4. Intervention Planning Phases 1-2. Select Health, Environmental & Behavioral Goals 2c. Select Intervention Approaches 2b. Select Channels and Mediators 2a. Select Intervention Objectives Policy Advocacy Community Leaders Healthful policies Phases 4-5:. Dissemination & Implementation Community Development Community Norm Shapers Healthful Communities Community Organization Organization Decision-Makers Healthful Organizations To Influence Individuals Individuals at Risk Healthful behavior Health Status 7. Conduct Impact Evaluation 8. Conduct Outcome Evaluation 6. Conduct Process Evaluation Phases 6-8. Evaluation *Adapted from Simons-Morton B, Greene W, & Gottlieb N. Introduction to Health Education & Health Promotion, 2nd ed. Waveland Press, 1995, p.154.

9. From Planning to Implementation & Evaluation Chapter 5 Phase 3 Planning PRECEDE Phases 4+ Intervention Alignment Administrative & Policy Diagnosis Predisposing Communications: Health Education Training Media Advocacy Behavior & Lifestyle Reinforcing Policy Regulation Organization Enabling : Environment Evaluation: Process………...Impact……Outcome--> PROCEED

10. ALIGNING INTERVENTIONS WITH EVIDENCE, THEORY, EXPERIENCE, POLICY, IMPLEMENTATION AND FORMATIVE EVALUATION Chapter 5: TURNING THE CORNER FROM PRECEDE to PROCEED Green & Kreuter, Health Program Planning: An Educational and Ecological Approach. 4th ed. NY: McGraw-Hill, 2005; and building on Green & Lewis, Measurement & Evaluation in Health Education and Health Promotion. Palo Alto: Mayfield Publishing Co., 1984.

11. Best Practices vs. Best Processes • Best practices are interventions recommended on the basis of systematic reviews of evidence from controlled studies that substantiate their efficacy in the situations in which they were tested, but not necessarily their effectiveness in other populations and circumstances. • Best processes are methods such as those of PRECEDE and the matching of evidence-based interventions to align and adapt them to the needs of a particular population and setting. Green & Kreuter, Health Program Planning: An Educational and Ecological Approach. 4th ed. NY: McGraw-Hill, 2005.

12. Emergence of a Sedentary Society 800 k-calories/day decrease in 20 years Millions of Person-Miles in Automobiles, 1970-1990 60% of kids walked to school in 1970. 13% do now 3000 2200 Hours of TV Viewed Daily 1960-1992 Percent of Workforce in Sedentary Occupations, 1950-1996 Source: Schroeder, 2003

13. Change in Per Capita Cigarette ConsumptionCalifornia & Massachusetts vs Other 48 States, 1984-1996 5 0 -5 Percent Reduction -10 -15 -20 -25 Other 48 States California Massachusetts 1984-1988 1990-1992 1992-1996

14. http://www.cdc.gov/tobacco

15. The Adaptation Process The Adaptation Guide, CDC, Div of HIV/AIDS Prevention., 2010. www.cdc.gov/hiv/topics/evaluation/health_depts/guidance/strat-handbook/chapter4.htm

16. Sources & Uses of Evidence & Theory in Population-Based Planning & Evaluation Models* 1. Assess Needs & Capacities of Population Evidence from community or population Evidence from Etiologic Research 2. Assess Causes (X) & Resources Reconsider X 4. Evaluate Program Evidence from Efficacy Studies, and Use of Theory to Fill Gaps 3. Design & Implement Program Program Evidence & Effectiveness Studies, and use of Theory *Green & Kreuter, Health Program Planning. 4th ed. NY: McGraw-Hill, 2005, Fig. 5-1.

17. Yarbrough, D.B., Shulha, L.J., Hopson, R.K., and Caruthers, F.W. (2011). The program evaluation standards: A guide for evaluators and evaluation users (3rd ed.). Thousand Oaks, CA: Sage. Program Evaluation Standards http://www.jcsee.org http://www.jcsee.org/program-evaluation-standards/program-evaluation-standards-statements

18. Public Health Core Functions and Essential Services Community Tool Box http://ctb.ku.edu

19. Evaluation is…... … the systematic assessment of the operation and/or outcomes of a program or policy, compared to a set of explicit or implicit standards as a means of contributing to the improvement of the program or policy Weiss, p4

20. The Program Evaluation Standards • Key features • Standards identify and define evaluation quality • Guide evaluators and evaluation users in pursuit of evaluation quality • “laws” vs voluntary, consensus • Revised 2011 • Clarifications • Now fifth standard of evaluation accountability • Trade-offs among standards

21. The Program Evaluation Standards • Utility-- The utility standards support high quality evaluation use through attention to all aspects of an evaluation • Feasibility -- The feasibility standards encourage evaluation to be effective and efficient. • Propriety-- The propriety standards are intended to ensure that an evaluation will be proper, fair, legal, right, acceptable, and just. • Accuracy-- Accuracy is the truthfulness of evaluation representations, propositions, and findings, which is achieved through sound theory, methods, designs, and reasoning. • Evaluation Accountability -- Documenting and improving evaluation accountability requires similar efforts to those required for program accountability, i.e., an evaluation of the evaluation (metaevaluation)

22. Evaluations judge the program;Standards judge the evaluation

23. Utility Standards The utility standards support high quality evaluation use through attention to all aspects of an evaluation • U1: Evaluator credibility • U2: Attention to stakeholders • U3: Negotiated purposes • U4: Explicit values • U5: Relevant information • U6: Meaningful processes & products • U7: Timely & appropriate communicating & reporting • U8: Concern for consequences and influence

24. Feasibility Standards • F1: Project management • F2: Practical procedures • F3: Contextual viability • F4: Resource use The feasibility standards are intended to ensures that an evaluation will be realistic, prudent, diplomatic, and frugal

25. Propriety Standards The propriety standards are intended to ensure that an evaluation will be conducted legally, ethically, and with due regard for the welfare of those involved in the evaluation, as well as those affected by its results • P1: Responsive & inclusive orientation • P2:Formal agreements • P3:Human rights & respect • P4: Clarity and fairness • P5:Transparency & disclosure • P6: Conflicts of interest • P7:Fiscal responsibility

26. Accuracy Standards …intended to ensure that an evaluation will reveal and convey technically adequate information about the features that determine worth or merit of the program being evaluated. • A1: Justified conclusions and decisions • A2: Valid information • A3: Reliable information • A4: Explicit program & context descriptions • A5: Information management • A6: Sound design & analysis • A7: Explicit evaluation reasoning • A8: Communication & reporting

27. Evaluation Accountability Standards Accountability refers to the responsible use of resources to produce value. In program evaluation, this requires an evaluation of the evaluation. • E1: Evaluation documentation • E2: Internal metaevaluation • E3: External metaevaluation

28. Trade offs Among Standards • Utility vs accuracy • Accuracy vs Feasibility • Feasibility vs utility • Propriety vs nothing

29. Evaluation Standards: So What? • Power of knowing what counts as “good” evaluation • Know where to find standards: ww.jcsee.org • Rationale for explaining evaluation trade-offs • Accountability