1 / 26

S519: Evaluation of Information Systems

S519: Evaluation of Information Systems. Result D-Ch10. Putting them together. Now we are ready to write our evaluation report. Basically we are going to fill our content to the checklist boxes we learned in lec2. The Key Evaluation Checklist (D-p. 6-7). I. Executive Summary.

sharlenew
Download Presentation

S519: Evaluation of Information Systems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. S519: Evaluation of Information Systems Result D-Ch10

  2. Putting them together • Now we are ready to write our evaluation report. • Basically we are going to fill our content to the checklist boxes we learned in lec2.

  3. The Key Evaluation Checklist (D-p. 6-7) I. Executive Summary II. Preface III. Methodology 1. Background & Context 2. Descriptions & Definitions 3. Consumers 4. Resources 5. Values 10. Exportability 7. Outcome Evaluation 6. Process Evaluation 8 & 9. Comparative Cost-Effectiveness 11. Overall Significance 14. Reporting & Follow-up 12. Recommendations & Explanations 13. Responsibilities 15. Meta-evaluation

  4. KEC Preliminary checkpoints • To orient readers to the basics of : • what the evaluation is and • why the evaluation was done • What the main approach was

  5. I. Executive summary • Shows a short overview (1-2 pages) of what was evaluated and what the main findings were. • You can consider to include the followings: • Very short description of the program • The overall conclusion about the quality or value of the evaluation • The graphical profile of the evaluand‘s performance (see Exhibit7.6) • Several (<7) bullet points about the most important strengths and weaknesses.

  6. II. Preface • Address the following questions: • Who asked for the evaluation and why? • What are the main evaluation questions? • Is this a formative or summative evaluation? • Who are the main audiences for this evaluation report? • Mainly based on your work in Ch2 • Please always provide justifications for your conclusions

  7. III. Methodology • The main methodologies used in the evaluation • How you set up your experiments • How you collect data • How you analyze your data? • How you draw conclusion • And each with why.

  8. KEC foundations checkpoints • To represent all the initial ingredients • Background • Context • What it is • Whom it serves • The nature and limitation of any resources • Where the values come from

  9. 1. Background and Context • Incude just enough information for readers to be able to understand the basic retionale for the program • Contents are: • Why did this program or product come into existence in the first place (needs assessment) • How is the evaluation supposed to address the needs and problems? • What are the main context of evaluands (e.g. Physical, economic, political) to facilitate or constrain the evaluation?

  10. 2. Description and definitions • Describe your evaluand in enough details • Definition of important terms

  11. 3. Consumers • Identify the actual and potential recipients or users of the evaluand • Describe consumers in geographic location, demographics • Downstream impactee

  12. 4. Resources • Describe the available and important resources (which helps to understand the conclusion of the evaluation) • Funds/budget • Physical space • Experience • Networks

  13. 5. Values • To convey to the audience how the evaluation team determinded what should be considered as „valuable“ or „high quality“ for the evaluand. • Justify the validity

  14. KEC Sub-evaluation checkpoints • Getting evaluation explicitly

  15. 6. Process evaluation • The issues of evaluation, content and implementation (see Ch4) • Ethics, the priniciple of equity and fairness • Consistency with professional and scientific standards • Efficiency (i.e, minimal wasted effort or resource) • Needs of consumers (e.g., timeliness, learning style, current knowledge) • Needs of staff (e.g., performance on tasks and activities)

  16. 7. Outcome evaluation • The outcomes are what happened to the consumers as a result of coming into contact with the evaluand • Intended and unintended • Short-term and long-term effects • A good way to do • List the main outcome dimensions with a brief explanation of how the list was generated. • Rate each dimension on importance • Explain the used methods (ch7) • Rate the each outcome dimension (ch8) • Provide the justification (evidence, logic, methods) for your rate

  17. 8&9. Comparative cost-effectiveness • It is important to know whether the evaluand is cost-effective • See D-p62-63

  18. 10. Exportability • Whether your evaluation could also have value outside its current context. • It is always worth considering the possibility

  19. KEC conclusions checkpoints • Combining Checkpoints6-10 to draw an overall conclusion

  20. 11. Overall significance • Using synthesis methodology to draw a summary covering Checkpoints 6-10 • Graphical representation (e.g., Exhibit7.6) • Summary of strengths and weaknesses

  21. 12. Recommendations and explanations (possible) • Recommendations for furture improvement

  22. 13. Responsibilities (possible) • Who is repsonsible for good or bad results • You have to be highly skilled

  23. 14. Report and support • Who get the copies of the evaluation reports and in what forms • What are follow-ups to ensure that the findings are used

  24. 15. Meta-evaluation • Self review of the evaluation itself • The validity of their conclusions • Utility to relevant stakesholders • The methodology used for evaluations • Credibility • Cost • Lessoned learned during this evaluation.

  25. Exercise • Go back to your evaluation group • Discuss all these checkpoints and how are you going to put all your result together

  26. Exercise • Go through these checkpoints • Write down 3 checkpoints which are most important to you and state why? • Discuss with your partner • Form the group and select 3 most important checkpoints, why and • Present your group conclusion and your suggestions in poster • Hang your group poster and people can shop around poster • Vote for the best poster

More Related