1 / 25

Evaluation 101

Evaluation 101. Prevention Education Meeting May 29, 2013. What is Evaluation?. Definitions of Evaluation. Examine and judge carefully. Determine value or worth.

halla-rodriquez
Download Presentation

Evaluation 101

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Evaluation 101 Prevention Education Meeting May 29, 2013

  2. What isEvaluation?

  3. Definitions of Evaluation • Examine and judge carefully. • Determine value or worth. • Process that attempts to determine, systematically and objectively, the relevance, effectiveness, and impact of activities in light of their objectives.

  4. Purpose of Evaluation • Use evidence to prove that a program has delivered intended services. • Show that program has made an impact on the health of a certain population. Windsor, Clark, Boyd, & Goodman. Evaluation of Health Promotion, Health Education, & Disease Prevention Programs

  5. Purpose of Evaluation • To determine achievement of objectives related to improved health status • To improve program implementation • To provide accountability to funding agencies, community, and other stakeholders McKenzie, Neiger, & Smeltzer. Planning, Implementing, & Evaluating Health Promotion Programs

  6. Outputs • The amount of product or service that the program intends to provide • Type of activity, product or service • andtarget audience Participants Clients Agencies Decision makers Customers Public Health care providers • Type of evaluation • process evaluation • Can also include satisfaction with activity/product/service

  7. Short Term Outcomes • Immediate effects of activities including changes in Awareness Knowledge Attitudes Beliefs Values Opinions Aspirations Motivation Skills (knowledge) • Type of evaluation • impact evaluation

  8. Intermediate Outcomes • Changes in behavior Risk factor reduction Norms Self management Policies Social action Practice Procedures Decision-making Demonstration of skills • Type of evaluation • impact evaluation

  9. Long Term Outcomes • Changes in burden of disease, health status, or quality of life • prevalence or incidence • mortality • also social, economic, civic, environmental factors • Type of evaluation • outcome evaluation

  10. Data Sourcesfor Evaluation

  11. Evaluation Terminology • Process evaluation • measurement of activities during implementation • Impact evaluation • measurement of immediate observable effects of a program including awareness, knowledge, attitudes, skills, and behaviors • Outcome evaluation • measurement of the ultimate goal of a program

  12. Process Evaluation Data • Quantitative • Counts • # meetings held, # training sessions conducted, # of people who attended • Descriptive Statistics • mean satisfaction rating for event, percent of people who responded they liked the event • Qualitative • Descriptive information • description of event, date of event, location of event, description of who attended, what information was included during event

  13. Process Evaluation Data • Monthly Progress Report • Example: • # professional training sessions/workshops conducted • # professionals trained • # informational materials/brochures distributed • # primary prevention education sessions conducted for students • # students attended primary prevention education sessions • # informational materials/brochures distributed • # sessions conducted with faith-based organizations • descriptions of topics covered during sessions

  14. Impact Evaluation Data • Short term outcomes on a logic model • awareness, knowledge, attitude change data • Quantitative • data collected from surveys/questionnaires, written tests • Qualitative • data collected from surveys/questionnaires, interviews, focus groups

  15. Impact Evaluation Data • Intermediate term outcomes on a logic model • behavior change data • Quantitative • BRFSS data • Qualitative • interviews, focus groups, observations

  16. Impact Evaluation Data • Example: • The results of the pre/post knowledge tests indicted that participants’ knowledge about healthy relationships increased by 500%. • The results indicated that students learned to use negotiation techniques as assessed by post-test. • Responses on the modeling questionnaire indicate that 85% of students would intervene when other students act disrespectfully. • School administrators at 10 schools strengthened their anti-bullying policies.

  17. Outcome Evaluation Data • Long term outcomes Changes in health status • Quantitative • YRBS data • BRFSS data • Vital data • Hospital Discharge data • ER or ED data • Qualitative • Interviews, focus groups

  18. Outcome Evaluation Data • Example: • 2007 YRBS • 29.9% of high school students had been in a physical fight at least once in the past year • 11.8% of high school students had been hit by their boy/girlfriend in the past year • 9.5% of high school students have been forced to have sexual intercourse against their will • 2008 BRFSS • 12.5% of adult females have been victims of forcible rape • 16.7% of adult females have either been raped or have experienced an attempted rape • 4.7% of adult males have either been raped or have experienced an attempted rape

  19. Evaluation Toolsand Resources

  20. ABCDE Method of Writing Outcomes • A – Audience (Who will change?) • B – Behavior (What will change?) • C – Condition (By When?) • D – Degree (How Much?) • E – Evidence (How will the change be measured?)

  21. Evaluation Activity • Developing a data collection tool

  22. General Evaluation Guides • The Program Managers Guide to Evaluation • http://www.acf.hhs.gov/programs/opre/other_resrch/pm_guide_eval/reports/pmguide/pmguide_toc.html • Kellogg Foundation Logic Model Development Guide • http://www.wkkf.org/Pubs/Tools/Evaluation/Pub3669.pdf • Developing a Logic Model: Teaching and Training Guide • http://www.uwex.edu/ces/pdande/evaluation/pdf/lmguidecomplete.pdf • Understanding Evaluation: The Way to Better Prevention Programs • http://www.ed.gov/PDFDocs/handbook.pdf • Kellogg Foundation Evaluation Handbook • http://www.wkkf.org/Pubs/Tools/Evaluation/Pub770.pdf

  23. Topical Evaluation Guides • Core Performance Measures for Steps to a HealthierUS • http://www.cdc.gov/eval/steps/Steps%20Core%20Performance%20Measures%2007-2007.pdf • Introduction to Program Evaluation for Comprehensive Tobacco Control Programs • http://www.cdc.gov/tobacco/tobacco_control_programs/surveillance_evaluation/evaluation_manual/00_pdfs/Evaluation.pdf • Physical Activity Evaluation Handbook • http://www.cdc.gov/nccdphp/dnpa/physical/health_professionals/interventions/handbook.pdf • CDC DASH Evaluation Briefs • http://www.cdc.gov/HealthyYouth/evaluation/resources.htm#4 • Strategic Planning Kit for School Health Programs • http://www.cdc.gov/HealthyYouth/evaluation/pdf/sp_kit/sp_toolkit.pdf • SMART Cards • http://www.cdc.gov/HealthyYouth/evaluation/pdf/SMARTcards.pdf

  24. Violence Prevention Evaluation Guides • Sexual and Intimate Partner Violence Prevention Programs Evaluation Guide • Book by CDC, 2007, can be ordered from • http://wwwn.cdc.gov/pubs/ncipc.aspx • Measuring Violence-Related Attitudes, Behaviors, and Influences Among Youths: A Compendium of Assessment Tools, Second Edition • http://www.cdc.gov/ncipc/pub-res/pdf/YV/YV_Compendium.pdf

  25. Evaluation Websites • CDC’s Evaluation Framework • http://www.cdc.gov/eval/framework.htm • Community Tool Box • http://ctb.ku.edu/en/ • American Evaluation Association • http://www.eval.org/ • Coalitions Work • http://coalitionswork.com/ • Writing Good Goals and Objectives • http://apps.nccd.cdc.gov/dashoet/writing_good_goals/menu.html • Basic Guide to Program Evaluation • http://www.managementhelp.org/evaluatn/fnl_eval.htm • University of Wisconsin Extension – Program Development and Evaluation • http://www.uwex.edu/ces/pdande/index.html • Kellogg Foundation • http://www.wkkf.org/default.aspx?tabid=75&CID=281&NID=61&LanguageID=0

More Related