EVAL 6000: Foundations of Evaluation

1. EVAL 6000: Foundations of Evaluation Dr. Chris L. S. Coryn Kristin A. Hobson Fall 2011

2. Agenda • American Evaluation Association conference reflections and experiences • Stage Three theories • General characteristics • Lee Cronbach • Questions and discussion • Encyclopedia of Evaluation entries

3. General Characteristics of Stage Three Theories • Synthesis of Stage One and Stage Two theories • Circumstantial, tailored, situational approach to evaluation • Theories that are contingent • Denial that there exist only one right way to do evaluation

4. Stage Three Theory of Social Programming • Programs are politically affected • Programs change gradually • Improving existing programs offers the best chance to contribute to short-term social change • Abrupt and novel changes do sometimes occur

5. Stage Three Theory of Knowledge Construction • Recognized that no single paradigm has sufficient empirical or theoretical support to dominate • Evaluation should embrace multiple epistemologies, methods, and priorities for the types of knowledge that are important • Dependent upon circumstances

6. Stage Three Theory of Valuing • Reliance on stakeholder values complemented by independent sources of values • Use of multiple sources of relevant values so that important values are not overlooked • Separate summative conclusions by each criterion rather than over all criteria

7. Stage Three Theory of Knowledge Use • Emphasis on enlightenment as well as instrumental use • Enlightenment is more likely to be the focus of those who are not under pressure to respond to requests for specific information • Instrumental use is more likely to be the focus of those who are placed to facilitate such uses

8. Stage Three Theory of Evaluation Practice • Specified contingencies under which different evaluation practices should be used and make sense • Often centered around the various stages of program lifecycles • Contingencies

9. “It is self-defeating to aspire to an evaluative conclusion as precise and…as beyond dispute…as from the laboratory” — Lee J. Cronbach

10. Biographical Sketch • Born in 1916 in Fresno, California • Died in 2001 in Palo Alto, California • Ph.D. in Education Psychology, University of Chicago • Scored 200 on Stanford Binet at age 5 (by some accounts at age 4) • Completed high school at age 14 • Graduated from Fresno State College at age 18 with degrees in mathematics and chemistry

11. Cronbach’s View of Evaluation • Mainly a critic of prior intellectual traditions and offered recommendations for improvement • Recognition of the lack of strong substantive theories and powerful methods of inquiry • Greater attention to external validity and/or generalizability

12. Cronbach’s Influence • Far reaching • Social science (social inquiry) methodology (at least equal to, and in some respects greater than, Campbell) • Measurement theory • Individual differences • Program evaluation • Epistemology

13. Cronbach’s Major Contributions • Social inquiry • Political accommodation • Generalizability theory • Generalization through explanation • Aptitude-treatment interactions • Bandwidth and fidelity • utos, UTOS, *UTOS, sub-utos

14. Cronbach’s Theory of Social Programming • Rejection of naïve normative expectations (e.g., rational decision making) • The core of social programing and policy making is among conflicting stakeholders • Assumes four stages: (1) breadboard; (2) superrealization; (3) prototype; and (4) ongoing program

15. Cronbach’s Theory of Knowledge Construction • Knowledge is predicated on assumptions regarding determinants (i.e., sources of variance) of performance (and, therefore, knowledge claims) • No clear epistemological position, though multiple methods are clearly advocated

16. Cronbach’s Theory of Valuing • Prescriptive valuing plays only a small role • Descriptive valuing is central • Favored a democratic control that assigned open debate as central to value claims

17. Cronbach’s Theory of Knowledge Use • Enlightenment rather than instrumental use is preferable • Use occurs not only from evaluation findings, but also through prior knowledge and from basic research about substantive theories • It is unrealistic to expect that knowledge will dictate decisions • Long-term view to influence future programs

18. Cronbach’s Theory of Evaluation Practice • Constructed around assumptions about types of questions worth asking and the degree of certainty worth attaining • Eclectic in choice of methods, avoiding adherence to any particular methods • Educator rather judge, philosopher-king, or servant to a particular stakeholder group

19. Encyclopedia Entries • Cousins, J. Bradley • Logic Model • Patton, Michael Quinn • Participatory Evaluation • Program Logic • Program Theory • Rossi, Peter H. • Utilization-Focused Evaluation