1 / 10

Brainstorming Session

The Kenneth and Anne Griffin Foundation. Experimentation as a tool: wider and more practical perspectives. Min Lee, Director . Brainstorming Session. May 16, 2010. 1. August 15, 2012. Outline. www.griffin-foundation.org Experimentation as a tool for program evaluation

kaden
Download Presentation

Brainstorming Session

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Kenneth and Anne Griffin Foundation Experimentation as a tool: wider and more practical perspectives Min Lee, Director Brainstorming Session May 16, 2010 1 August 15, 2012

  2. Outline • www.griffin-foundation.org • Experimentation as a tool for program evaluation • Running experiments does not need to be complex • The benefits of thinking about experimentation • Additional thoughts August 15, 2012 2

  3. Experimentation as a tool for program evaluation • Why is it important? • Organization providing after school programs to seniors in a high school • Provides support in math and English • Students: • higher scores in state standardized assessments in math and English • higher high school graduation rate • compared to school district averages August 15, 2012 3

  4. Experimentation as a tool for program evaluation August 15, 2012 4

  5. Experimentation as a tool for program evaluation • Question: is the school district the appropriate comparison group? • Had the seniors not participated in the after school program, what would have their math/English scores and graduation rates been? • Answer: we don’t know • What’s driving the higher scores/graduation rates? The program or something else? • Why is this kind of questions important? • Gets to the fundamental issue of its effectiveness • Accountability to constituents and stakeholders • Applicable to other organizations: Charter school, food pantry, services to alcoholics/drug addicts, etc. • Experimentation can help us answer these questions August 15, 2012 5

  6. Running experiments does not need to be complex • Some experiments can be complex (in particular those looking to innovate) • Non-profit organizations can benefit enormously from simple experiments if they are able to: • Randomize people into treatment/control groups • Able to measure/track the relevant variables • Example: after school program • More seniors want to participate than spaces available • Randomly select those who will participate and those who will not • Track both groups in terms of math and English scores and high school graduation rate August 15, 2012 6

  7. Running experiments does not need to be complex • Example: organization that relies on volunteers to serve the constituents • Volunteer retention is critical and your manager has different initiatives that would increase retention rate • Don’t implement all the initiatives across all volunteers • For a smaller group, test the initiatives by randomizing volunteers into them • Measure the retention rates relative to the cost of the initiatives August 15, 2012 7

  8. The benefits of thinking about experiments • Organizations seriously thinking about ‘effects’ and how to measure them • Organizations thinking seriously about ‘causation’ • Organizations thinking seriously about ‘selection bias’ August 15, 2012 8

  9. Additional thoughts • Partnerships with researchers • Take advantage of the experimental economists • Training/hiring staff members to think about these issues August 15, 2012 9

  10. Concluding remarks • Successful in convincing you that… • Experimentation is not an esoteric tool used by scientist • BUT a tool that is relevant to your organization • Your organization can also run experiments and learn from the results to better serve your constituents • Even if you cannot run experiments… • There are benefits from being SERIOUS about cause/effect and proper measurement/comparison • Need to have staff members to be able to think about these issues… • BUT don’t need econ Ph.Ds August 15, 2012 10

More Related