EVAL 6000: Foundations of Evaluation

1. EVAL 6000: Foundations of Evaluation Dr. Chris L. S. Coryn Kristin A. Hobson Fall 2011

2. Agenda • Stage One theories • Donald T. Campbell • Questions and discussion • Encyclopedia of Evaluation entries

3. “We would improve program evaluation if we were alert to opportunities to move closer to the experimental model” — Donald T. Campbell

4. Biographical Sketch • Born in 1917, died in 1996 • Ph.D. in Psychology, University of California, Berkeley • Author or more than 235 publications • Recipient of numerous honorary degrees, awards, and prizes • Intellectual work included psychological theory, methods, sociology of science, and epistemology

5. Campbell’s View of Evaluation • Evaluation should be a part of a rational society in which decisions depend on the results of rigorous tests of bold attempts to improve social problems • Evaluators should play a servant-methodologist role rather than an advisory role commensurate with democratic values

6. Campbell’s Influence • Lionized as the father of scientific evaluation • Developed and legitimated scientific methods of evaluation • The utopian view of an ‘experimenting society’

7. Campbell’s Major Contributions • Evolutionary epistemology • Validity theory and threats to validity • Experimental and quasi-experimental methods • Open, mutually reinforcing but critical commentary on knowledge claims (a disputatious community of truth seekers)

8. Randomized Experiments • Provide ‘best’ scientific evidence of cause-and-effect relationships • Premised on expectancy of equivalence of units through randomly assigning units to two or more conditions • Priority is to reduce internal validity threats

9. Flow of units through a typical randomized experiment

10. Campbell’s Theory of Social Programming • Three worlds • The current world: Client needs are not the driving force behind political and administrative behavior • The current world as it can be marginally modified: Improvement through demonstrations • The utopian world: Critical reality checks and the experimenting society

11. Campbell’s Theory of Knowledge Construction • Grounded in epistemological relativism (knowledge is impossible without active knowers) • Never knowing what is true and imperfectly knowing what is false • Evolutionary theory of knowledge growth • Not all methods yield equally strong inferences

12. Campbell’s Theory of Valuing • Valuing should be left to the political process, not researchers (descriptive valuing) • Evaluators are not the arrogant guardians of truth • Multidimensional measurement that is inclusive of democratic values

13. Campbell’s Theory of Knowledge Use • Use is the concern of the political process, not evaluators • Evaluations are only worth using if they have withstood the most rigorous tests • Most concerned with misuse • Methodological biases • Control of content or dissemination

14. Campbell’s Theory of Evaluation Practice • Application of experimental design to answer summative questions • Priority given to internal validity • Theoretical explanation is best left to basic researchers • Evaluation resources should be focused on pilot and demonstration projects

15. Encyclopedia Entries • Bias • Causation • Checklists • Chelimsky, Eleanor • Conflict of Interest • Countenance Model of Evaluation • Critical Theory Evaluation • Effectiveness • Efficiency • Empiricism • Independence • Evaluability Assessment • Evaluation Use • Fournier, Deborah • Positivism • Relativism • Responsive evaluation • Stake, Robert • Thick Description • Utilization of Evaluation • Weiss, Carol • Wholey, Joseph