Evaluation Result

1. Evaluation Result

2. Putting them together • Now we are ready to write our evaluation report. • Basically we are going to fill our content to the checklist boxes we learned in lec2. S519

3. The Key Evaluation Checklist (D-p. 6-7) I. Executive Summary II. Preface III. Methodology 1. Background & Context 2. Descriptions & Definitions 3. Consumers 4. Resources 5. Values 10. Exportability 7. Outcome Evaluation 6. Process Evaluation 8 & 9. Comparative Cost-Effectiveness 11. Overall Significance 14. Reporting & Follow-up 12. Recommendations & Explanations 13. Responsibilities 15. Meta-evaluation S519

4. KEC Preliminary checkpoints • To orient readers to the basics of : • what the evaluation is and • why the evaluation was done • What the main approach was S519

5. I. Executive summary • Shows a short overview (1-2 pages) of what was evaluated and what the main findings were. • You can consider to include the followings: • Very short description of the program • The overall conclusion about the quality or value of the evaluation • The graphical profile of the evaluand‘s performance (see Exhibit7.6) • Several (<7) bullet points about the most important strengths and weaknesses. S519

6. II. Preface • Address the following questions: • Who asked for the evaluation and why? • What are the main evaluation questions? • Is this a formative or summative evaluation? • Who are the main audiences for this evaluation report? • Mainly based on your work in Ch2 • Please always provide justifications for your conclusions S519

7. III. Methodology • The main methodologies used in the evaluation • How you set up your experiments • How you collect data • How you analyze your data? • How you draw conclusion • And each with why. S519

8. KEC foundations checkpoints • To represent all the initial ingredients • Background • Context • What it is • Whom it serves • The nature and limitation of any resources • Where the values come from S519

9. 1. Background and Context • Incude just enough information for readers to be able to understand the basic retionale for the program • Contents are: • Why did this program or product come into existence in the first place (needs assessment) • How is the evaluation supposed to address the needs and problems? • What are the main context of evaluands (e.g. Physical, economic, political) to facilitate or constrain the evaluation? S519

10. 2. Description and definitions • Describe your evaluand in enough details • Definition of important terms S519

11. 3. Consumers • Identify the actual and potential recipients or users of the evaluand • Describe consumers in geographic locatoin, demographics • Downstream impactees S519

12. 4. Resources • Describe the available and important resources (which helps to understand the conclusion of the evaluation) • Funds/budget • Physical space • Experience • networks S519

13. 5. Values • To convey to the audience how the evaluation team determinded what should be considered as „valuable“ or „high quality“ for the evaluand. • Justify the validity S519

14. KEC Sub-evaluation checkpoints • Getting evaluation explicitly S519

15. 6. Process evaluation • The issues of evaluation, content and implementation (see Ch4) • Ethics, the priniciple of equity and fairness • Consistency with professional and scientific standards • Efficiency (i.e, minimal wasted effort or resource) • Needs of consumers (e.g., timeliness, learning style, current knowledge) • Needs of staff (e.g., performance on tasks and activities) S519

16. 7. Outcome evaluation • The outcomes are what happened to the consumers as a result of coming into contact with the evaluand • Intended and unintended • Short-term and long-term effects • A good way to do • List the main outcome dimensions with a brief explanation of how the list was generated. • Rate each dimension on importance • Explain the used methods (ch7) • Rate the each outcome dimension (ch8) • Provide the justification (evidence, logic, methods) for your rate S519

17. 8&9. Comparative cost-effectiveness • It is important to know whether the evaluand is cost-effective • See D-p62-63 S519

18. 10. Exportability • Whether your evaluation could also have value outside its current context. • It is always worth considering the possibility S519

19. KEC conclusions checkpoints • Combining Checkpoints6-10 to draw an overall conclusion S519

20. 11. Overall significance • Using synthesis methodology to draw a summary covering Checkpoints 6-10 • Graphical representation (e.g., Exhibit7.6) • Summary of strengths and weaknesses S519

21. 12. Recommendations and explanations (possible) • Recommendations for furture improvement S519

22. 13. Responsibilities (possible) • Who is repsonsible for good or bad results • You have to be highly skilled S519

23. 14. Report and support • Who get the copies of the evaluation reports and in what forms • What are follow-ups to ensure that the findings are used S519

24. 15. Meta-evaluation • Self review of the evaluation itself • The validity of their conclusions • Utility to relevant stakesholders • The methodology used for evaluations • Credibility • Cost • Lessoned learned during this evaluation. S519

25. Exercise • Go back to your evaluation group • Discuss all these checkpoints and how are you going to put all your result together S519