340 likes | 455 Views
Enhanced Implementation of CDC Guidelines for School Tobacco Programs: Results of a Statewide Evaluation. Shelly A. Greller, MS Wisconsin Department of Public Instruction D. Paul Moberg, Ph.D University of Wisconsin – Madison. Acknowledgements. Wisconsin Department of Public Instruction
E N D
Enhanced Implementation of CDC Guidelines for School Tobacco Programs:Results of a Statewide Evaluation Shelly A. Greller, MS Wisconsin Department of Public Instruction D. Paul Moberg, Ph.D University of Wisconsin – Madison
Acknowledgements • Wisconsin Department of Public Instruction • John Hisgen • Douglas White • University of WI – Monitoring & Evaluation Program (www.medsch.wisc.edu/mep) • David Ahrens • Ann Christiansen • Amy Anderson • Patrick Remington, P.I. • Funded by the Wisconsin Tobacco Control Board
Participants should be able to: • Describe WI statewide school tobacco grant program • Describe CDC Guidelines for School Health Programs to Prevent Tobacco Use and Addiction • Identify effective program resources and strategies • Understand & apply results of WI school tobacco program to enhance implementation of CDC Guidelines • Apply assessment & evaluation tool to operationalize CDC Guidelines
Background • Master Settlement Agreement led to creation of WI Tobacco Control Board (WTCB) • WTCB developed comprehensive strategic plan for tobacco control • One focal area was youth tobacco prevalence • Youth goal: By 2005, tobacco use among middle & high school age youth will decline by 20% • School-based programs funded for $1,250,000 by competitive process through DPI
Background • DPI stated purpose for school-based tobacco control programs • “Create or expand upon strategies identified as most promising or effective in reducing or eliminating youth tobacco use” • Operationally, discussion focused on increased implementation of CDC Guidelines
Monitoring and Evaluation • Monitoring and Evaluation Program (MEP) contracted to perform monitoring/evaluation functions for WTCB • MEP comprised of • UW – Comprehensive Cancer Center • UW – Extension • Center for Health Policy and Program Evaluation • MEP collaborated with DPI on evaluation of school-based tobacco control programs
CDC Guidelines • Guidelines for School Health Programs to Prevent Tobacco Use and Addiction (MMWR, 1994) • Summarizes school based strategies “most likely to be effective in preventing tobacco use by youth” • Developed by CDC in collaboration with experts • In-depth review of research, theory & current practice
Impact of Guidelines • Rohde et al. (2001 in MMWR) • High or medium implementation of Guidelines in Oregon associated with significantly greater decline in 8th graders’ 30-day smoking prevalence
Impact of Guidelines • Rohrbach et al.(2002 Presentation at Society for Prevention Research-SPR) • 2 waves of student data from sample of schools • Indexed Guideline Implementation based on teacher surveys • Most schools did not fully implement comprehensive program of evidence-based strategies • Guideline implementation significantly related to smoking prevalence - quit attempts - negative expectations/attitudes regarding tobacco
Impact of Guidelines • Hallfors & Godette (2002, HER) • Dept. of Education has recent statement of “Principles of Effectiveness” for prevention programs • Surveyed 104 School Districts in 12 states • Only 19% of school districts are implementing research-based prevention curricula with “fidelity”
Goal of Our Evaluation • Focus on institutional change in implementation of the guidelines in schools with infusion of targeted funding • Not on impact of implementation of Guidelines on student outcomes
Methods of Our Evaluation Design Sample: Applicant School Buildings/Districts
Measures – Self Assessment • Each building in applicant district completed checklist operationalizing CDC Guidelines • Operationalizes CDC Guidelines with 58 questions • Original purpose was self assessment and planning • Program evaluation secondary • Repeated at end of year 1 and year 2 • Funded districts submitted as part of year end report • Unfunded districts mailed as survey at end of year 1
Scaling/Data Reduction • Exploratory Principal Components factor analysis indicated most items within areas of the guidelines were correlated • We created additive indices for each of the 7 areas • Scaling 0 = No 1 = Somewhat 2 = Yes
Scale Validation • 2001-2002 School Health Education Profile (SHEP) oversampled to include all funded schools • Reports from SHEP will be correlated with reports from schools • Compared baseline data of funded schools to full SHEP sample • How representative are they?
Mean Change During Year 1 *P < .05 **P < .001
Mean Change During Year 1 *P < .05 **P < .001
Mean Change During Year 1 *P < .05 **P < .001
Regression Results Regression analysis covarying baseline value found: • Baseline covariate always a sig. predictor of one year implementation measure • Significant effects of funding on • Training: B = .155 (.084) • Family / community involvement: B = .162 (.072) • Cessation: B = .25 (.09)
Variation by Level – Funded Schools • Elementary schools (57 schools) • High on baseline family & community involvement • Lowest on evaluation • Middle schools (52 schools) • Highest overall score at baseline, especially in training and curriculum • High schools (30 schools) • More change (than elementary & middle schools) on policy, training, parent & community involvement & cessation • Highest initially on cessation • Combinations (16 schools) • Highest level of change overall: high change on policy, curriculum, and instruction
Limitations and Issues • Tool originally intended as self assessment • Not a research tool • Validity/reliability to be established • SHEP link will help – in progress • Self Report from school staff (who want funding to continue) without validation • Low response rate from comparison group • No student outcomes
Conclusions • Baseline implementation of Guidelines highest in areas of • Policy • Curriculum • Implementation of Guidelines lowest in • Cessation • Evaluation • Training • Family / community involvement
Conclusions • Most gain with funding in • Cessation • Evaluation • Training • Family / community involvement • Unfunded schools also report gain in all areas--but significantly less in several areas • The tool is promising to assess school tobacco control programs both for planning & evaluation