Evaluation Research, aka Program Evaluation

1. Evaluation Research, aka Program Evaluation

2. Definitions • Program Evaluation is not a “method” but an example of applied social research. • From Rossi and Freeman: “Program Evaluation is the systematic application of scientific methods to assess the design, implementation, improvement or outcomes of a program.” • Put more succinctly, program evaluation (or “evaluation research”) determines if programs “work.”

3. Dimensions of Program Evaluation • Process Evaluation: Is the (ongoing) program being conducted as was intended? • Outcome Evaluation: Did the (completed) program have the effect that was intended?

4. Stages of Program Evaluation • Needs Assessment: Assesses nature of problem and what has to be “solved.” • Program Theory: Formal description of the program, ie, how it addresses needs. • Process Analysis: Evaluates if program theory is being implemented properly, eg, staffing and clientele outreach assessments. • Impact Analysis: Examines effects of programs via causal analysis of some sort. • Cost-Benefit Analysis: Determines economic efficiency of the program.

5. “Impact Analysis” and the Role of Experimentation • Program Evaluation can in principle partake of any research method… • However, the logic of program evaluation and its perception of “programs” as having causal effects on outcomes calls for an experimental orientation • This need is evident even if the “experiment” isn’t laboratory based and even where survey or field-study data are collected, because for PE controlled comparisons are important.

6. Some examples of Program Evaluation • Criminal justice (especially juvenile delinquency) programs • Public health interventions • Education, eg the U of C’s podcasting experiment