330 likes | 339 Views
This webinar explores the significance of evaluation in state oral health programs, including the status of evaluation in these programs and the key factors associated with program success. It also discusses the essential public health services for promoting oral health and the expectations of funders regarding evaluation. Participants will gain access to practical resources for conducting evaluations in public oral health.
E N D
Resources for State Oral Health Program Evaluators January 30, 2013 • B. J. Tatro, PhD • ASTDD, bj@bjtatro.com Cassandra Martin Frazier, MPH, CHES CDC, bkx9@cdc.gov
General Reminders • This webinar will be recorded and archived on the ASTDD website • Questions will be addressed after the speakers are finished. Please type your question into the “chatbox” that will appear at the end of the webinar and then click on the bubble to the right of where you type your question to send it to the moderator • Please respond to the polling questions at the conclusion of the webinar
This presentation was supported by Cooperative Agreement 5U58DP001695 from CDC, Division of Oral Health. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of CDC
Desired Results • Increase awareness of the importance of evaluation and evaluation capacity within state oral health programs • Increase familiarity with the scope of public oral health evaluation topics • Increase familiarity with and access to a variety of practical and useful evaluation resources relevant to public oral health
PART 1 Importance of Evaluation and Evaluation Capacity in Public Oral Health
Evaluation: A Key Component of State Oral Health Program Infrastructure
Status of Evaluation in State Oral Health Programs • In 2012, ASTDD completed a report on the status and factors impacting state oral health infrastructure. The capacity to conduct evaluation was one of these. Findings included the following: • According to CDC, the number of funded states reporting using evaluation to improve programs had increased from 3 to 11 of the 12 by the end of their second state grant funding cycle • In a 2009 ASTDD survey, most states indicated that they had access to evaluation expertise, but 13 did not • Currently there is no way to assess and track if and how non-CDC funded states are evaluating their programs, partnerships, or policies
Key Factors Associated with Program Success/Improvements in Oral Health “Evaluation helps build infrastructure and enhance sustainability when results are used to improve programs, increase program visibility and demonstrate program achievements. Strong programs have evidence-based goals, conduct routine evaluation and alter their programs based on evaluation results.” (ASTDD, State Oral Health Infrastructure and Capacity: Reflecting on Progress and Charting the Future, 2012)
Evaluation Capacity Evaluation helps build infrastructure but the capacity to do evaluation must be present Evaluation requires: • Sufficient and appropriate staffing • Ongoing training • Access to data • Access to decision makers • Financial resources
Importance of Evaluation in Public Oral Health In the ASTDD Guidelines for State and Territorial Oral Health Programs, there are 10 Essential Public Health Services to Promote Oral Health in the US, one of which is: Evaluate effectiveness, accessibility, and quality of personal and population-based oral health promotion activities and oral health services
State Roles in Public Oral Health Evaluation • Create a comprehensive evaluation plan • Evaluate availability, utilization, appropriateness, and effectiveness of activities and services, with partners • Provide evaluation consultation to partners • Validate and share results with community members, stakeholders, and other partners • Use results in oral health improvement and planning • Leverage and allocate sufficient resources for evaluation
Essential Public Health Services to Promote Oral Health in the U.S.
Evaluation of Other Essential Public Oral Health Services Virtually all of the other nine Essential Services have an evaluation component!
Funder Expectations for Evaluation • Federal Government:“A central pillar of good government is a culture where answering such [outcome and impact] questions is a fundamental part of program design and where agencies have the capacity to use evidence to invest more in what works and less in what does not.” (Program Evaluation, in Budget of the United States Government, Fiscal Year 2011: Analytical Perspectives, 2010, 90–92. Washington, DC: U.S. Government Printing Office) • Foundations: “The field of philanthropy is undergoing a fundamental transition toward more performance-centered and forward-looking evaluation approaches that provide foundations and their grantees with timely information and actionable insights.” (Mark Kramer, From Insight to Action: New Directions in Foundation Evaluation, April 2007)
CDC Structure for Evaluation • Describe the program • Develop logic models • Develop a comprehensive evaluation plan and objectives • Document what happens in the program • Maintain accurate and organized records on programs • Designate a person responsible for evaluation • Create success stories • Improve the program • Use evaluation findings to plan program improvements • CDC’s Framework for Program Evaluation in Public Health
CDC Structure for Evaluation • Incorporate evaluation into the following required activities for CDC-funded states: • Leadership • The surveillance system • The state oral health plan • Partnerships and state oral health coalitions • Sealant programs • Policy development
Competencies for Public Oral Health Evaluators CDC has identified competencies for State oral health evaluators
Competencies for Public Oral Health Evaluators Competencies based on: • Guiding Principles for Evaluators, American Evaluation Association • CDC, Framework for Program Evaluation • Stevahn L, King J, Ghere G, Minnema J. Establishing Essential Competencies for Program Evaluators.American Journal of Evaluation,2005; 26 (1) (43-59) • Yarbrough, D, Shulha, L, Hopson, R, and Caruthers, F (2011). The Program Evaluation Standards: A Guide for Evaluators and Evaluation Users (3rd ed.). Thousand Oaks, CA: Sage • ASTDD Competencies for State Oral Health Programs
PART 2 Resources for Public Oral Health Evaluators
Where to Find New Resources • ASTDD website (slides from this webinar will also be posted on the ASTDD website) • CDC evaluation website • Join the American Evaluation Association (AEA) or a local affiliate • AEA Linked-In • University of Wisconsin Extension Centerevaluation website • Western Michigan University, the Evaluation Center • Online Evaluation Resource Library – National Science Foundation website • Bureau of Justice Assistance evaluation website • Conferences, including the AEA Annual Conference, National Oral Health Conference and preconference workshops, and CDC/AEA Summer Institute • Online and live trainings, including AEA, CDC, and colleges/universities
Next Steps • Are you interested in: • Future evaluation webinars? • NOHC evaluation roundtables? • NOHC evaluation presentations? • NOHC evaluation pre-sessions? • A evaluation community of practice? • An evaluation blog? • Are you willing to post evaluation articles, data sources, tools, or other resources to a shared site, if it were available?