EVAL 6000: Foundations of Evaluation

1. EVAL 6000: Foundations of Evaluation Dr. Chris L. S. Coryn Kristin A. Hobson Fall 2011

2. Agenda • Stage Two theories • General characteristics • Carol Weiss • Activity • Questions and discussion • Encyclopedia of Evaluation entries

3. General Characteristics of Stage Two Theories • Characterized as a period of enlightenment in the evolution of evaluation theory • Argued rigorous causal knowledge was not always useful for improving programs • Emphasized different users of evaluation and their roles in “valuing” social programs • Focused on complex nature of social programs

4. Stage Two Theory of Social Programming • Recognized “political” aspects of evaluation • Directed efforts toward “incremental change” • Sought to determine “who” could make changes

5. Stage Two Theory of Knowledge Construction • Priority was given to “truthful and useful” • Took a pragmatic stance – willing to sacrifice valid knowledge for useful knowledge • Reflected predominately as methodological pluralism (with the exception of Stake) • More thorough understanding of social programs “in context” • Central focus was “discovery” related concerns

6. Stage Two Theory of Valuing • Predominately advocated descriptive values • Reflected the notion of “pluralistic” sources of values as valuing takes place in a political context of social programs with multiple stakeholders and constituents • Prefer to let intended users determine value • Valuing is not a task for evaluators given the complexities involved in valuing as well as the complexities involved in decisions about social programs

7. Stage Two Theory of Knowledge Use • Dominant concern • Who would use? • For what? • Concerns reflected • In a view of wider types of uses than those anticipated by Scriven and Campbell • Concerted efforts to identify users • Patton’s ideological perspective (i.e., intended users for specific uses) • Users and uses differed between major stage two theorists

8. Stage Two Theory of Evaluation Practice • Emphasis on methodological pluralism • Wholeyand Stake assigned central methods to practice • Weiss never claimed particular methods were to be preferred over others

9. “It does appear that evaluation research is most likely to affect decisions when the researcher accepts the values, assumptions, and objectives of the decision-maker” — Carol H. Weiss

10. Biographical Sketch • Ph.D. in Sociology and M.A. in Government, Columbia University • B.A. in Government, Cornell University • Beatrice B. Whiting Professor at the Harvard University Graduate School of Education • Awards • Howard Davis Memorial Lecture Award, Knowledge Utilization Society (1991) • Policies Studies Organization Award (1988) • Alva and Gunnar Myrdal Award for Science, Evaluation Research Society (1980) • Written 11 books

11. Weiss’s View of Evaluation • Gave priority to state and federal decision makers in her theory of evaluation • Originally premised on truth, later developed a contingency-based theory • Who? • For what purpose? • Later in her career she became more concerned with enlightenment

12. Weiss’s Influence • Developed some of the earliest ideas about “theory-based evaluation” • Started the enlightenment phase • What it means to use evaluation

13. Weiss’s Major Contributions • Linked evaluation to policy research • Recognition that evaluation as political dimensions • Types of use, and what it means to use evaluation • Instrumental • Truth • Utility • Decision accretion • Development of a theory-based approach to understanding the effects of evaluation

14. Weiss’s Theory of Social Programming • Programs are guided by political, stakeholder, and organizational interests • Evaluator does not provide solutions but provides stakeholders with better understandings so they can design their own solutions

15. Weiss’s Theory of Knowledge Construction • Knowledge is built over time • Direct knowledge is rarely used in decision making • Useful knowledge is more than truth • Ontological and epistemological concern arose

16. Weiss’s Theory of Valuing • Descriptive • Stakeholders values should dominate the evaluation • Reflects the political context of the program and social problem solving • Program can be good if meets specified levels of performance • Each conclusion can stand alone • Decision-makers can assign their own weight

17. Weiss’s Theory of Knowledge Use • Favors enlightenment use over instrumental use • Focuses on providing results for use by decision-makers (federal and state managers)

18. Weiss’s Theory of Evaluation Practice • Quantitative in orientation • Uses multiple methods • Fits the method to the problem • Choice of method related to • Types of knowledge to be generated • Intended users of that knowledge • Questions to be answered through evaluation • Focuses on enlightenment and development of information to allow decision-makers to make good choices • Later work, concerned with issues about generalization and explaining how programs work

19. Encyclopedia Entries • Conceptual use • Context • Countenance model of evaluation • Decision accretion • Developmental evaluation • Enlightenment • Evaluability assessment • Evaluation use • Goal • Implementation • Incremental change • Instrumental use • Intended users • Intended uses • Objectives • Performance indicators • Process use • Responsive evaluation • Setting • Stake, Robert • Stakeholders • Thick Description • Truth test of use • Uses of Evaluation • Utility test of use • Utilization of Evaluation • Weiss, Carol • Wholey, Joseph