EVAL 6000: Foundations of Evaluation

1. EVAL 6000: Foundations of Evaluation Dr. Chris L. S. Coryn & Carl D. Westine October 7, 2010

2. Stage 2 Theories • Stage 2 theories can be characterized as a period of “enlightenment” in the evolution of evaluation theory • Theorists argued that rigorous causal knowledge was not always useful for improving programs • Theorists emphasized different users of evaluation and their roles in “valuing” social programs • Theorists focused on the complex nature of social programs • Despite a number of commonalities, the three major theorists of this period differed in more and less significant ways

3. Stage 2 Theories • Criterion 1: Theory of Social Programming • Stage 1 theorists operated on “naive” assumptions (i.e., that radical changes would be made as a result of evaluation • Stage 2 theories and theorists recognized the “political” aspects of evaluation and directed efforts toward “incremental change” (Weiss, in her later work, reverted to more radical ideologies) • Related to theories of use, Stage 2 theories of social programming sought to determine “who” could make changes • The resulting assumptions were complex and produced no definitive solutions

4. Stage 2 Theories • Criterion 2: Theory of Use • In Stage 2, theories of use became the dominant concern of theories and theorists • Who would use? • For what? • These concerns were reflected in a view of wider types of uses than those anticipated by Scriven and Campbell • They also reflected concerted efforts to identify users • Ultimately, this became Patton’s ideological perspective (i.e., intended users for specific uses) • Even so, users and uses differed between the major theorists of the time

5. Stage 2 Theories • Criterion 3: Theory of Knowledge Construction • Priority shifted from “truth” to both “truthful and useful” • During this period theories and theorists took a “pragmatic” stance • Willingness to sacrifice valid knowledge for useful knowledge • This pragmatic position was reflected in various ways, but predominately as “methodological pluralism” (with the exception of Stake) • This position also included a more thorough understanding of social programs “in context” • Central focus was no longer effects, but included a variety of more “discovery” related concerns

6. Stage 2 Theories • Criterion 4: Theory of Valuing • Theorists at this stage predominately advocated descriptive values • This view reflected the notion of “pluralistic” sources of values as valuing takes place in a political context of social programs with multiple stakeholders and constituents • Here, the three dominant theorists of the period prefer to let intended users determine value • Valuing is not a task for evaluators given the complexities involved in valuing as well as the complexities involved in decisions about social programs

7. Stage 2 Theories • Criterion 5: Theory of Practice • Although methodological pluralism was prominant during this period, Wholey and Stake assigned central methods to practice • Weiss never claimed that particular methods were to be preferred over other • Her choice of method was one related to types of knowledge to be generated, intended users of that knowledge, and the questions to be answered through evaluation • In her later work, she (like Cronbach) became more concerned with issues about generalization and explaining how programs work

8. Carol Weiss • Weiss gave priority to state and federal decision makers in her thoery of evaluation • Originally premised on “truth” (similar to Stage 1 theories) she later developed a contingency-based theory • Who? • For what purpose? • Later in her career she became more concerned with “enlightenment”

9. Joseph Wholey • Wholey was predominately concerned with management issues and decisions reflecting their concenrs • Like Weiss, Wholey began under the auspises of “truth” seeking • His later work emphasized “instrumental use” of evaluation—immediate decisions and actions • Overall, his central concern was producing actionaable information aimed at those who are in positions to take decisions • Like Weiss, his choice of method was quantitative in orientation

10. Robert Stake • Stake, unlike prior (and later) theorists, developed his theory on the premise of “discovery” • By discovery, Stake implied that the only realities (i.e., truth) are those that are constructed • He also claimed that valuing is socially constructed • The value of a program is best determined by those who experience it • Emphasis was on local stakeholders values • Premised on the assumption that most decision making takes place locally, and that values of local constituents often compete with others

11. Contrasting Stage 1 and 2 • Work with the person sitting to your right, select one of the criteria for evaluation theory, and critically compare and contrast (using example if possible) Stage 1 and Stage 2 theorists’ position on the selected criterion • Theories of Social Programming • Theories of Use • Theories of Knowledge Construction • Theories of Valuing • Theories of Practice

12. Further Discussion of the Readings • What specific questions were raised as you read the assigned readings for this week (or past weeks)? Is there something specific that you didn’t understand? • What problems remain unresolved regarding Stage 2 theories? • What other issues emerged from the readings that need further exploration?

13. Encyclopedia Entries for this Week • Conceptual use • Context • Countenance model of evaluation • Cronbach, Lee J. • Developmental evaluation • Evaluability assessment • Evaluation use • Goal • Implementation • Intended users • Intended uses • Objectives • Performance indicator • Process use • Responsive evaluation • Rossi, Peter H. • Setting • Stake, Robert E. • Thick description • Utilization of evaluation • Weiss, Carol H. • Wholey, Joseph S.