1 / 35

Low Cost and No Cost Tools for Program Evaluation

Low Cost and No Cost Tools for Program Evaluation. Victor Balaban, PhD Program Evaluation Team (PET) Field Services and Evaluation Branch (FSEB) Division of Tuberculosis Elimination (DTBE) NCHHSTP/CDC. Disclaimer.

tocho
Download Presentation

Low Cost and No Cost Tools for Program Evaluation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Low Cost and No Cost Tools for Program Evaluation Victor Balaban, PhD Program Evaluation Team (PET) Field Services and Evaluation Branch (FSEB) Division of Tuberculosis Elimination (DTBE) NCHHSTP/CDC

  2. Disclaimer • The contents and conclusions in this presentation have not been formally disseminated by CDC and should not be construed to represent any agency determination or policy.

  3. Evaluation • Evaluation is: • An activity that assists in planning and measuring programs • a way of managing, improving and being accountable for: • resources • activities • results • Evaluation answers the question- “Is the program doing what we intend it to do?”

  4. What Can Be Evaluated? • Direct service interventions • Community mobilization efforts • Research initiatives • Surveillance systems • Policy development activities • Outbreak investigations • Laboratory diagnostics • Communication campaigns • Infrastructure-building projects • Training and educational services • Administrative systems Source: MMWR, 1999, Framework for Program Evaluation in Public Health

  5. Why is Evaluation Important? • Improve knowledge of program design • Improve program implementation • Reporting • Ensure that a program reaches those who need it most • Give visibility to work • Demonstrate accountability • Share information • Enhance understanding of what works best and what does not work – and why

  6. Low Cost and No Cost Tools for Program Evaluation • When designing an evaluation, there are often existing tools that can be used • Many were not designed for evaluating TB programs and so will need to be modified • Focus on evaluation instruments and tools for data collection and analysis, not guidelines and trainings

  7. CDC Resources

  8. Find TB Resources • http://www.findtbresources.org/programevaluation.aspx • Resources: • Links to tools and resources related to program evaluation

  9. CDC - Office of the Associate Director for Program (OADPG) • http://www.cdc.gov/eval/resources/index.htm#datacollection • Resources: • Step by Step Manuals • Manuals/Assistance with Specific Evaluation Steps: • Logic models • Data Collection Methods and Sources • Websites Offering Comprehensive Evaluation Resources and Assistance

  10. Evaluation Guidance Handbook: Strategies for Implementing the Evaluation Guidance for CDC-Funded HIV Prevention Programs • http://www.cdc.gov/hiv/topics/evaluation/health_depts/guidance/strat-handbook/appendix.htm • Instruments: • Intervention plan worksheets • Data collection forms

  11. CDC - Healthy Youth! Program EvaluationStrategic Planning Tools & Resources • http://www.cdc.gov/healthyyouth/evaluation/sp_toolkit.htm#3 • Strategic Planning Kit for School Health Programs: • Data Sources Matrix • Identifying Stakeholders Worksheet • SWOT Worksheet for School Health Programs • Program Strategy Checklist • Program Communication Tracking Tool • Data Collection Framework

  12. CDC - Healthy Youth! Handbook for Evaluating HIV Education • http://www.cdc.gov/HealthyYouth/publications/hiv_handbook/index.htm • Instruments: • HIV and AIDS knowledge related to HIV-risk behaviors • Attitudes toward people with AIDS • Attitudinal dimensions related to HIV-risk behaviors • Confidence in one's ability to resist peer pressures • Perceptions of peers' behaviors and values • Intentions regarding HIV-risk behaviors and their precursors • HIV-risk behaviors

  13. TB-PEN Resources

  14. TB-PEN – Tools Committeehttp://tbpen.pbworks.com/w/page/4124488/FrontPageInstruments: (organized by National TB Objectives) • Completion of Treatment • TB Case Rates • Contact Investigations • Laboratory Reporting • Treatment Initiation • Sputum Culture Conversion • Sputum Culture Reporting  • Data Reporting • Recommended Initial Therapy • Universal Genotyping • Known HIV Status • Evaluation of Immigrants and Refugees   • Program Evaluation • Human Resource Development Plan • TB Training Focal Points  • Cohort Review  

  15. American Evaluation Association (AEA) Resources

  16. AEA – Online Resources • http://www.eval.org/Resources/instruments.asp • Links to collections of evaluation instruments • http://www.eval.org/Resources/surveylinks.asp • Links to companies that provide software for the development, distribution, scanning, and analysis of surveys/questionnaires

  17. AEA 365 • http://www.aea365.org/blog/ • A blog dedicated to highlighting Tips, Tricks, Resources, and Lessons Learned for evaluators

  18. Other Resources

  19. Children, Youth and Families Education and Research Network (CYFER) • http://www.cyfernet.org/index.php?c=6 • Instruments for: • Process Evaluation - populations served, satisfaction, program accountability, etc. • Program Sustainability • Organizational Capacity

  20. UCSF - Center for AIDS Prevention Studies (CAPS) • http://caps.ucsf.edu/research/survey-instruments/ • Instruments for: • Evaluation • Treatment adherence • Focus group guides

  21. Innovation Network – Point K Learning Center • http://www.innonet.org/index.php?section_id=4&content_id=16 • Instruments: • Organizational Assessment Tool • Logic Model Builder • Evaluation Plan Builder

  22. Penn State Extension: Program Evaluation • http://extension.psu.edu/evaluation/examples.html • Resources: • Evaluation tipsheets that answer questions about program evaluation • Evaluation examples, including selections from instruments

  23. University of Wisconsin - Extension • http://www.uwex.edu/ces/pdande/evaluation/evalinstruments.html • Instruments: • Community Organization Assessment Tool • Program Planning Questionnaire

  24. The Measurement Group • http://www.themeasurementgroup.com/evaluationtools/evaluation_tools.htm • Instruments • General Purpose Instruments and Evaluation Modules Evaluation of the HRSA SPNS Innovative Models of HIV/AIDS Care • Evaluation Report Forms for the JAHF Institute for Geriatric Nursing • General Purpose Instruments and Specialized Evaluation Materials for Evaluation of the HRSA SPNS Adolescent HIV/AIDS Care Models • Metro DC Collaborative for Women with HIV/AIDS Needs Assessment Instruments • Evaluation Instruments for the HIFY Empowered Youth Evaluation (EYE) Model

  25. Wilderdom • http://www.wilderdom.com/tools/ToolsIndex.html • Instruments • List of 30+ tools for assessing the psycho-social impacts of intervention programs e.g. program quality surveys • Summary reviews of the main features of each instrument, with overall rating and links to further information. • Links to the major sites on the web for finding more research instruments.

  26. DoView • http://www.doview.com/ • DoView is a software program that enables evaluators to create logic models

  27. Data Collection and Analysis

  28. Data Collection and Analysis • CDC Resources • Epi-Info • EZ-Text • Google Resources • Google Analytics • Google Refine

  29. Epi-Info • http://wwwn.cdc.gov/epiinfo/index.htm • Software for the rapid creation of quantitative data collection instruments, analyzing and visualizing data, and generating reports • Creates questionnaires and data entry forms • Automatically creates a database from the questionnaire • Reads and analyzes data into statistics, tables, graphs, and maps • Generates reports

  30. EZ-Text • http://www.cdc.gov/hiv/topics/surveillance/resources/software/ez-text/index.htm • Software program to assist in creating, managing, and analyzing qualitative data • Interview responses are entered into EZ-Text as a verbatim transcript (e.g., from a tape recording), or a summary generated from the interviewer's notes

  31. EZ-Text (cont.) • http://www.cdc.gov/hiv/topics/surveillance/resources/software/ez-text/index.htm • Users can investigators can interactively create on-line codebooks, apply codes to specific response passages, develop case studies, conduct database searches to identify text passages, and export data for further analysis with other qualitative or statistical analysis software programs

  32. Google • Google Analytics • http://www.google.com/analytics/index.html • A free program that help to analyze traffic to websites and measure the success of social media programs • Google Refine • http://code.google.com/p/google-refine/ • for data cleanup and transformation to other formats to help clean messy data

  33. Conclusion • Using and adapting existing instruments can save time and resources when conducting evaluations • DTBE/FSEB/PET is available to provide information to help you identify and use evaluation tools

  34. Thank you

  35. Acknowledgements • DTBE/FSEB/Program Evaluation Team (PET) • Awal Khan • Judy Gibson • Lakshmy Menon • Brandy Peterson • DTBE/FSEB/Field Services Teams I & II • Dan Ruggiero • Gail Burns-Grant • Vincent Fears • Alstead Forbes • Regina Gore • Vernard Green • Andy Heetderks • Mark Miner • Dawn Tuckey

More Related