1 / 23

Perspectives on Impact Evaluation Cairo, Egypt March 29 – April 2, 2009 Presented by:

A Model for Using Technical Assistance to Improve the Evaluation Capacity of Local Programs. Perspectives on Impact Evaluation Cairo, Egypt March 29 – April 2, 2009 Presented by:

shandi
Download Presentation

Perspectives on Impact Evaluation Cairo, Egypt March 29 – April 2, 2009 Presented by:

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A Model for Using Technical Assistance to Improve the Evaluation Capacity of Local Programs Perspectives on Impact Evaluation Cairo, Egypt March 29 – April 2, 2009 Presented by: Wayne M. Harding. Ed.M., Ph.D. , Director of Projects, Social Science Research & Evaluation, Inc., Burlington, MA USA wharding@ssre.org Cheryl Vince-Whitman, M.B.A., Ed.M., Senior Vice President, Education Development Center, Newton, MA USA cvincewhitman@edc.org

  2. Agenda • Brief Overview of Service to Science (STS). • Evaluation Findings About Service to Science • Key Characteristics of The Technical Assistance Provided By Service To Science

  3. Overview of Service to ScienceInitiative

  4. The Need for Service-to-Science • Federal agencies and others in the USA have promoted the use of evidence-based programs to prevent substance abuse. • The supply of “proven” programs is too limited to meet many local conditions. • There are many other programs that may work, but lack evidence of their effectiveness

  5. Service-to-Science Goals • To help innovative prevention interventions that address substance abuse (or related issues) develop, improve, and document evidence of their effectiveness. • To increase the pool of effective and appropriate interventions.

  6. Organization of Service-to-Science • Funded by Center for Substance Abuse Prevention (CSAP) in 2004. • Implemented through 5 regional prevention centers (Centers for the Application of Prevention Technologies – CAPTs). • In 2007, CSAP funded a related initiative for programs designed for Native Americans. Implemented by the Native American Center for Excellence (NACE).

  7. Service-to-Science Activities • CAPTs solicit prevention program nominations from State prevention officials. • CAPTs assess the readiness of candidate programs. • CAPTs conduct regional Service to Science Academies. • CAPTs provide follow-up technical assistance.

  8. Service-to-Science Activities (Continued) • Service to Science Academy participants – eligible to compete for “subcontract awards” to further enhance evaluation capacity. • CAPTs provide limited technical assistance post subcontract award.

  9. Number of Programs Served Through Service to Science • As of July 2009, the CAPTs had provided 294 prevention programs with technical assistance. • Since 2006, 110 programs have received the subcontracts of up to $30,000.

  10. Selected Evaluation Findings

  11. Methods • Pilot study (2006) interviews and/or online survey with program Directors. Target sample of 79 programs; 58 (73%) responded. • Six case studies of 2005 STS participants and extended follow-up case studies on 5 that received mini-subcontracts. Six case studies of 2006 participants. • 2007 online survey targeted 142 programs that had been participants for at least 15 months. 93 responded (a 77% rate after removing 21 programs for which respondents were no longer available).

  12. Participant ProgramCharacteristics

  13. Primary Behaviors Targeted for Change (a) Multiple responses allowed. N=93.

  14. Ages of Population(s) Served (a) Multiple responses allowed. N=93.

  15. Race/Ethnicity of Population(s) Served (a) Multiple responses allowed. N=93.

  16. Strategies Used (a) Multiple responses allowed. N=93.

  17. Service To ScienceOutcomes

  18. Value of STS Services (a) N=93.

  19. Factors That Facilitated Progress When asked to discuss factors that facilitated progress on their evaluation the largest percentage of clients (79%) identified TA from the CAPTs. (a) From the 2006 pilot study. N=58.

  20. TA Contributed to Program Capacity(a) in 5 Most Common Issues (a) Contributed “A Little,” “Somewhat,” or “A Great Deal” vs. “Did Not Contribute at All.” (b) N=93.

  21. Seeking Recognition and/or Funding (a) N=93.

  22. Defining Features Of Service To Science TA

  23. Defining Features of Service to Science TA • Client-centered • Customized • Long-term • Relationship-based • Flexible

More Related