Management-Oriented Evaluation

1. Management-Oriented Evaluation …evaluation for decision-makers. Jing Wang And Faye Jones

2. Theoretical Basis—Systems Theory • Systems Theorist—Burns and Stalker (1972), Azumi and Hage (1972), Lincoln (1985) , Gharajedaghi (1985), Morgan (1986). • Systems theorist in education—(Henry Bernard, Horace Mann, William Harris, Carleton Washburne). • Mechanical/linear constructions of the world versus organic/systems constructions. • Closed versus Open systems. The role of the environment. References: Patton, 2002: Fitzpatrick, Sanders, & Worthen, 2004: Scott, 2003.

3. Theoretical Basis • Systems Theory Reference: Scott, 2003.

4. Defining and conceptualizing a system • A system is a whole that is both greater than and different from its parts • The effective management of a system requires managing the interactions of its parts, not the action of its parts taken separately (Gharajedaghi and Ackoff, 1985). Describe that system—volunteers! Reference: Patton, 2002.

5. Management Oriented Evaluation • The primary focus of management oriented evaluation is to serve the decision-maker (s). • The needs of the decision-makers guide the direction of evaluation.

6. Stufflebeam’s CIPP • CIPP serves decision-makers facing four types of decisions: Context Evaluation—planning decisions Input Evaluation—structuring decisions Process Evaluation—implementation decisions Product Evaluation—recycling decisions

7. Context Evaluation

8. Input Evaluation

9. Process Evaluation

10. Product Evaluation

11. Stufflebeam’s Evaluation Steps • Focusing the evaluation • Collection of information • Organization of information • Analysis of information • Reporting of information • Administration of the evaluation

12. Alkin’s UCLA Model • Types of evaluation • Systems assessment (context) • Program planning (input) • Program implementation • Program improvement (process) • Program certification (product)

13. Strengths • Focuses on informational needs and pending decisions of decision-makers • Systematic and comprehensive • Provides a wide variety of information • Stresses importance of utility of information • Evaluation happens throughout the program’s life • Provides timely feedback and improvement • CIPP—heuristic tool that helps generate important questions to be answered. Easy to explain.

14. Weaknesses • Narrow focus • Inability to respond to issues that clash with concerns of decision makers • Indecisive leaders unlikely to benefit • Possibly unfair or undemocratic evaluation • May be expensive and complex • Unwarranted assumptions • Important decisions may be correctly identified up front • Orderliness and predictability of decision-making process

15. Application of CIPP • Evaluation framework for nursing education programs: application of the CIPP model • Critical success factors • Create an evaluation matrix • Form a program evaluation committee including representatives from all partners • Determine who will conduct the evaluation: internal or external • Ensure the evaluators understand and adhere to the program evaluation standards

16. It’s Time for….. • Management Oriented Evaluation Trivia. • Split up into two groups. • Each group selects a team captain. • The group with the most money wins.

17. References • Fitzpatrick, J.L., Sanders, J.R., & Worthen, B.R. (2004). Program evaluation: alternative approaches and practical guidelines. White Plains, NY: Longman. • Patton, M. Q. (2002). Qualitative evaluation and research methods (3rd ed.). Newbury Park, California: Sage. • Scott, W. R. (2003). Organizations: Rational, natural, and open systems (5th ed.).Upper Saddle River, NJ: Prentice Hall. • Singh, M.D. (2004). Evaluation framework for nursing education programs: Application of the CIPP model. International Journal of Nursing Education Scholarship, 1, Issue 1.