1 / 12

S519: Evaluation of Information Systems

Understand and apply meta-evaluation principles to assess the quality and value of evaluations. Explore key criteria like validity, utility, conduct, credibility, and cost. Engage in practical exercises to evaluate and present results effectively, learning important lessons and discovering improvement opportunities.

dustinwoods
Download Presentation

S519: Evaluation of Information Systems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. S519: Evaluation of Information Systems Meta-evaluation D-Ch11

  2. Last week • What are these checkpoints? • How to put your result together? • What are lessons-learned during the exercise?

  3. Meta-evaluation • It is evaluation of an evaluation (Scriven, 1991) to determin the quality and/or value of an evaluation

  4. Five criteria • Validity • Utility • Conduct • Credibility • Costs

  5. Validity • To what extent, the conclusions of the evaluation are justified • Check carefully the facts they use and values they assign • Question everything • Constraints (budget, time, etc.) • Using KEC checkpoints • Using standards (www.wmich.edu/evalctr/jc/)

  6. Validity • Covers all relevant sources of value • Comprehensively covers process, outcome and cost • Include no irrelevant or illicit criteria • Data used to directly address the criteria • Include analyses that are appropriate for the data • Clearly states how data are interpreted • Is clear about where evaluative conclusions come from • Include valid recommendation

  7. Utility • Are the findings • Relevant to the questions or decisions being faced by the audience • Timely • Clearly communicated • Cost effective

  8. Conduct • Legal • Ethical • Professional standards • Cultrual appropriateness • Unobtrusiveness • Minimal disruptive to the evaluand

  9. Credibility • Familiar with the context • Independence, impartiality, and/or lack of conflict of interest • Expertise in evaluation and in the evaluation subject field

  10. Cost • How reasonable the costs of an evaluation

  11. Meta-evaluation rating table • Table 11.1

  12. Exercise • D-p217, exercise 2 • Form a pair, do the evaluation • Form a group to select the best result, • Design a poster • Present them in a poster • Using three sentenses to summarize your evaluation • What you learn • What you feel • How to improve

More Related