Unit 1 Module 4

1. Unit 1 Module 4 Explain: Roles of the Evaluator Introduction to Educational Evaluation 5100-652 Dr. Kristin Koskey

2. Multiple Personalities Grubbs, 2009; Mark, 2002; Patton, 2007; Skolits et al., 2009

4. Pre-Evaluation Stage • Role of the detective • Learn about the context • Conduct an evaluability assessment • Role of the negotiator • Reach an agreement on the evaluation • Role of the designer • Develop the evaluation plan

5. Pre-Evaluation Stage: Role of the Detective • Learn about the context • What is to be evaluated? • Who is the client? • Why was this program initiated? • What are the program goals? • What activities are believed to lead to these intended goals? • What is the theory behind these program goals • What is the political climate surrounding the evaluation? • Who are the most relevant stakeholders? • Who are the beneficiaries of the program? • Has this program been previously evaluated? If so, what have different stakeholders observed as a result of the program? • What data already exist on this program?

6. Pre-Evaluation Stage: Role of the Detective Continued • Determine the evaluability of the project • Identify whether the program model is clear. • Check the alignment between the program activities and program model • Determine if the program goals are achievable • Determine if the stakeholders information needs are feasible • Assess whether the stakeholders and you agree on the evaluation goals and uses

7. Pre-Evaluation Stage: Role of Detective Continued • When and when not to get involved

8. Pre-Evaluation Stage: Role of Detective Continued • When to and when not to get involved

9. Pre-Evaluation Stage: Role of Negotiator • Reach an agreement on: • Evaluation to be conducted • Goals of the evaluation • How it will be conducted • Timelines • Budget • Who will do what and when • Use of evaluation findings

10. Pre-Evaluation Stage: Role of Designer • Evaluation model(s) based on the evaluation goals • Research design • Methodology • Data sources to be collected • Formative and summative • Strategies for data analysis and reporting • Budget

11. Active-Evaluation Stage • Role of the diplomat • Build good working relationships and trust with the stakeholders • Role of the researcher • Implement the evaluation plan • Role of the judge • Program strengths and weaknesses • Value, merit, worth of whatever is being evaluated • Role of the reporter • Determine audience(s) • Communicate relevant evaluation results • Provide recommendations

12. Post-Evaluation Stage • Role of the advocator • Educate stakeholders how to use the evaluation results • Promote stakeholders use of the evaluation results • Role of the reflector • Engage in self-reflection as a professional • What worked well and what didn’t work well? • Is there anything I could have done differently? • What skills should I strengthen?

13. References • American Evaluation Association (2009). Guiding principles for evaluators. American Journal of Evaluation, 30(1), 5-6. • Fitzpatrick, J. L., Sanders, J. R., & Worthen, B. R. (2011). Program evaluation: Alternative approaches and practical guidelines (4th ed). Boston, MA: Pearson. • Grubbs, S. T. (2009). An evaluation of an alternative teacher certification program: A matter of trust. American Journal of Evaluation, 30(4), 581-586.  • Mark, M. M. (2002). Toward better understanding of alternative evaluator roles. In K. E. Ryan & T. A. Schwandt (Eds.), Exploring evaluator role and identity (pp. 17-36). Greenwich, CT: Information Age Publishing. • Patton, M. Q. (2007). Utlization-focused evaluation (4th ed.). Thousand Oaks, CA: Sage • Skolitis, G. J., Morrow, J. A., & Burr, E. M. (2009).Reconceptualizing evaluator roles. American Journal of Evaluation, 30(3), 275-295. doi: 10.1177/1098214009338872