Program Evaluation Overview

1. Program Evaluation Overview

2. Definitions of Program Evaluation • systematic collection of information abut the activities, characteristics, and outcome of programs • for use by specific people to reduce uncertainties, improve effectiveness, and make decisions with regard to what those programs are doing and affecting (Patton, 1986). 

3. Definitions of Program Evaluation • establish and provide useful information for judging decision alternatives; • assist an audience to judge and improve the worth of some educational program or object; • assist the improvement of policies and programs. Stufflebeam

4. Purpose of Program Evaluation • Improve • Maintain • Stop

5. Objectives of Program Evaluation • Discover whether and how effectively the objectives are being met • Discover unplanned and unexpected consequences • Determine underlying policies and related activities that contribute to success or failure in particular areas • Provide continuous feedback

6. Context, Input, Process, and ProductEvaluation The CIPP Model

8. CIPP Model • Simple systems model applied to program evaluation. • context evaluation, input evaluation, process evaluation, and product evaluation. • These types are typically viewed as separate forms of evaluation, but • they can also be viewed as steps or stages in a comprehensive evaluation

9. Formative < --------- > Summative With the CIPP model, there is a clear distinction between formative evaluation (conducted for the purpose of improving the program) and summative evaluation (conducted for the purpose of accountability which requires determining the overall effectiveness or merit of the program).

10. Context Evaluation • includes examining and describing the context of the program being evaluated, • conducting a needs and goals assessment, • determining the objectives of the program, and • determining whether the proposed objectives will be sufficiently responsive to the identified needs. It helps in making program planning decisions.

11. Input Evaluation • includes activities such as a description of the program inputs and resources, • a comparison of how the program might perform compared to other programs, • a prospective benefit/cost assessment (i.e., decide whether you think the benefits will outweigh the costs, before it is actually implemented), • an evaluation of the proposed design of the program, and • an examination of what alternative strategies and procedures should be considered and recommended. • In short, this type of evaluation examines what the program plans on doing. It helps in making program structuring decisions.

12. Process Evaluation • includes examining how a program is being implemented, • monitoring how the program is performing, • auditing the program to make sure it is following required legal and ethical guidelines, and • identifying defects in the procedural design or in the implementation of the program. • It is here that evaluators provide information about what is actually occurring in the program. • Evaluators typically provide this kind of feedback to program personnel because it can be helpful in making formative evaluation decisions (i.e., decisions about how to modify or improve the program). • In general, process evaluation helps in making implementing decisions.

13. Product Evaluation • includes determining and examining the general and specific outcomes of the program (using impact or outcome assessment techniques), • measuring anticipated outcomes, • attempting to identify unanticipated outcomes, • assessing the merit of the program, • conducting a retrospective benefit/cost assessment (to establish actual worth or value), and/or conducting a cost effectiveness assessment (to determine if the program is cost effective compared to other similar programs). • Product evaluation is very helpful in making summative evaluation decisions (e.g., What is the merit and worth of the program? Should it be continued?)