1 / 24

The Evaluation Process

The Evaluation Process. Nancy F. Barrett EdD February 12, 2014. ASSESS. PLAN. EVALUATE. DO. Definition. E valuation is the determination of the value or worth of a product, process or program . . Type Purposes . Identify new programs Reach new audiences Increase revenue

cleary
Download Presentation

The Evaluation Process

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Evaluation Process Nancy F. Barrett EdD February 12, 2014

  2. ASSESS PLAN EVALUATE DO

  3. Definition Evaluation is the determination of the value or worth of a product, process or program.

  4. Type Purposes Identify new programs Reach new audiences Increase revenue Make adjustments Increase efficiency Document goal attainment Report to funding agencies Needs Assessment Improvement Outcomes

  5. Another View Needs Outcomes Improvement Assessment Time Frame Future Present Past What COULD BE What IS What WAS Question Groteleuschen A (1980). “Program evaluation”. in Knox, A. (Ed). Developing, administering and evaluating adult education. San Francisco: Jossey-Bass.

  6. Other Important Considerations • Evaluation Question • Internal/External • Data Sources • Evaluation Plan • Data Analysis • Report

  7. Evaluation Question • Key to process • Focused • Concise • Measureable • Negotiated

  8. Internal Evaluation External Evaluation Hybrid

  9. Qualitative or Quantitative?? Quantitative Qualitative What Purpose Why Strength Process Measuring Outcomes Weakness Objectivity Context Use Improvement Outcome Attainment • Use Depends on • Evaluation Question • Availability (Usually some combination)

  10. Evaluation Plan “Road map” Connects questions to data sources Addresses validity and reliability Validity: Do findings address your question? Reliability: Are the findings stable over time? Multiple data sources are important

  11. Data Collection Methods Surveys • Forced choice • Open ended • Scaled response Focus groups Interviews Pre/post testing Observations • Obtrusive • Unobtrusive • Document review • Websites • Newsletters • Publications • Financial reports

  12. Data Analysis Considerations • Review data once it is collected. • Summarize each individual data source first. • Have someone else review work. • Make sure every data source addresses at least one evaluation question. • Look for themes. • Use tests of significance where appropriate.

  13. Evaluation SummaryUrbana Youth Summit May 12, 2012

  14. Program Origins • Low income, high crime area of Urbana • Need for youth programming

  15. Evaluation Design Evaluation Question: “How effective was the conference in identifying resources and programs for Urbana youth?” Outcomes/Needs Assessment Internal (ish) evaluation Intended data sources: • Survey • Qualitative data during conference • Observations • Content review of focus groups

  16. Implementation Problem Lack of Role Clarity Unfamiliarity with the structure Key Stakeholders left out Survey not implemented

  17. Any Outcomes? Yes! Enthusiastic participants Knowledgeable of the needs Identified problems and solutions

  18. Findings: Problems • Schools • Lack of community cohesion • Lack of resources • Housing, jobs • Recreation • Unsupervised time • Lack of “sense of self”

  19. Findings: Strengths • Academic and tutoring programs • Anti-bullying efforts in schools • Governmental support • Public Health initiatives • Neighborhood itself

  20. Findings: Suggestions • Youth programming • Community building activities • More family friendly businesses • Opportunities for parental involvement • Additional heath/nutrition programs • Job opportunities for youth • Better relationship with police • Volunteer/service opportunities

  21. Impact (Immediate) Youth Activities • City-funded summer program for youth 2. Community based summer arts program

  22. Impact (Longer Term) Summer arts program continuing for 3rd year Includes parent mentor component Healthy Kids initiative Community building efforts: Grant to train mentors for young men Advocacy with city around business development Community gardening efforts Discussing community safety with the police Continued city support

  23. ASSESS PLAN EVALUATE DO

  24. References Barrett Evaluations. http://www.barrett-evaluations.com/forms.html. Last accessed 2/7/2014. Barrett, N. (2013). Program Evaluation: A Step-by-Step Guide. Urbana IL: Sunnycrest Press. Barrett N. (2012) Urbana Youth Summit Evaluation. http://urbanaillinois.us/sites/default/files/attachments/youth-summit-summary_0.pdf Fitzpatrick J, Sanders J, and Worthen B. (2004). Program evaluation: alternative approaches and practical guidelines (3rd Edition). Boston: Allyn and Bacon. Groteleuschen A (1980). “Program evaluation” in Knox, A. (Ed). Developing, administering and evaluating adult education. San Francisco: Jossey-Bass.

More Related