1 / 12

Update on Planning the Next Generation of GEAR UP Evaluation Studies

Update on Planning the Next Generation of GEAR UP Evaluation Studies. 2009 NCCEP/GEAR UP Annual Conference July 19 2009. Three Topics --Update. Background and New Direction Update on RTI facilitated Design Work Discuss Planning Awards. GEAR UP First Evaluation.

ida
Download Presentation

Update on Planning the Next Generation of GEAR UP Evaluation Studies

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Update on Planning the Next Generation of GEAR UP Evaluation Studies 2009 NCCEP/GEAR UP Annual Conference July 19 2009

  2. Three Topics --Update • Background and New Direction • Update on RTI facilitated Design Work • Discuss Planning Awards

  3. GEAR UP First Evaluation • Evaluations mandated in authorization in 1998 • Westat began 10 year evaluation soon after program authorized—following cohort of 2000-01 7th graders through expected high school graduation year and fall entry into postsecondary • Quasi-experimental—matched comparison • Study analyses and reporting be concluding next year—Middle School report in 2007—High School report in early 2010

  4. Challenges and Issues From First Evaluation • Long period to obtain results/ study attrition • Serious Threats to validity (spill over; treatment/comparison non-equivalencies) • Black box study not telling us about specific practices • Not strong focus on program improvement

  5. New Approach to be Taken • Design next generation studies that will contribute to program improvement • Evaluation and Statistical Standards Based • Responsive to Congress and Accountability Efforts such as PART • Responsive to the needs of practitioners and students served

  6. What would we like the next generation of GEAR UP evaluations to look like? • Partnership (key stakeholders) • Standards Based (feasible, accurate, useful, ethical) • Practice (Grounded in understanding) • Reflection (analysis and synthesis of information from multiple sources-formative assessment) • Innovation/Improvement (start and end)

  7. Standards • Useful—to interested parties (congress, policy makers, practitioners, and ultimately students that the program is intended to benefit • Accurate--Scientific Rigor—evidence based • Feasible—possible to implement study • Proper—meet ethical standards for evaluation work

  8. Rigorous Evaluation Design Requirements • Counterfactual/comparison • Sample representative –external validity • Treatment and control or comparison group are equivalent on dimensions related to outcomes • Treatment and control/comparison group are treated equally except for intervention of interest • Treatment and control are mutually exclusive with regard to the intervention

  9. TRIO HEOA Evaluation Amendments • Prohibit ED requiring projects to deliberately recruit more students than they would normally serve and then denying service for study purposes • Call for working with the applicable institutions’ IRB’s • Call for rigorous studies focused on program improvement and addressing who can most benefit

  10. GEAR UP Design Work • How can we develop designs that are: useful, rigorous, feasible and proper? • What would such designs look like? • How should they best be structured for implementation? • How can we make the best use of the funds for evaluation provided for by Congress?

  11. What Are We Doing ? • Met with GEAR UP Project Director’s in Feb of 2008; July of 2008, February 2009—and now July 2009 • RTI Background work (profiles, systemic review, focus groups, expert papers) • Initiating Planning Awards Solicitation for GEAR UP projects (a little today and more on Monday)

More Related