1 / 20

How does your program work?

How does your program work?. Using program theories to guide your evaluation Mary S Marczak Research and Evaluation Specialist, University of Minnesota Extension. Theory of Life: Calvin as the Evaluator. Do you have a program theory?.

ludwig
Download Presentation

How does your program work?

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. How does your program work? Using program theories to guide your evaluation Mary S Marczak Research and Evaluation Specialist, University of Minnesota Extension

  2. Theory of Life: Calvin as the Evaluator

  3. Do you have a program theory? All programs whose objectives are to cause changes in participants are inherently theory based. Wilder Research Poll #1

  4. To what extent do you agree/disagree with the following statement? “The term theory as used by program developers and implementers typically is used to mean a vague notion or hunch not usually based on social science information.”  • Strongly Disagree • Tend to Disagree • Tend to Agree • Strongly Agree

  5. Program theory and evaluation • Building relationships and coming to a shared understanding • Test the program theory (Camp K, MN 4-H) • Formative evaluation (performance indicators, activities-theory fit, implementation quality) • Understand why programs do or do not work (theory failure? Or implementation failure?) • Attributing outcomes to the program • What else?

  6. Program theory: what is it? A plausible and sensible model of how a program is supposed to work L. Bickman, 1987

  7. A systematic delineation of cause and effect relationships M. Scheirer, 1987

  8. Identification of resources, activities, and outcomes of a program and the causal assumptions that connects these J. Wholey, 1987

  9. Scientific explanations of how social change is possible C. McClintock, 1987

  10. How should theory be constructed? Two camps • Program theory should be constructed by key program stakeholders (Wholey, Patton) • Program theory should stress social science theory and knowledge (Chen, Rossi) Poll #2

  11. In which “camp” do you put yourself as an evaluator? A) Theory should be constructed by program stakeholders B) Theory should be constructed based on social science theory and knowledge C) Some of both but more heavily influenced by stakeholders D) Some of both but more heavily influenced by social science theory E) They are equally important

  12. Pulling it together: Huey-tsyh Chen Two categories of theories that should be explicit in any program to be evaluated. Each have a role in terms of what we might evaluate. • Descriptive • Prescriptive

  13. Descriptive theories Assumptions about the nature of the problem and the population affected, limiting conditions, and what aspect of the problem should be solved.

  14. Descriptive theory example: http://eci.ext.msstate.edu/info/theory.pdf

  15. Prescriptive theories How to go about solving the problem or bringing about change; how should activities be constructed? How should activities/intervention be implemented? What is the causal connection to social change?

  16. Prescriptive theory example: J Baur and S Son, 2009 Powerpoint

  17. Pulling it all together example: Danes, 2009 “Staying Resilient in Times of Change” http://cehd.umn.edu/fsos/Centers/RuralMNLife/

  18. Defining the constructs- what are the key components, theory behind the construct, measures. Word Documents

  19. References • L. Bickman (ed.) Using program theory in evalulation. New Directions for Program Evaluation, no. 33. San Francisco: Jossey-Bass, Spring, 1987. (multiple authors) • Chen, H-T. Issues in constructing program theory. New Directions for Program Evaluation, no. 47, Fall 1990. • Chen, H-T and Rossi, P. H. The theory driven approach to validity. Evaluation and program Planning, 1987, 10, 95-103. • Patton, M. Q. A context and boundaries for theory-driven approach to validity. Evaluation and Program Planning, 1989, 12, 375-377. • Wilder Research Center. Program theories and logic models. St. Paul, MN: Wilder Research Center. http://chicano.umn.edu/pdf/resourcesProgram%20Theories%20Example%20Logic%20Model.pdf

More Related