1 / 18

Developing Evaluation Instruments

Developing Evaluation Instruments. Evaluations. Formative – How are we doing? Summative – How did we do? Confirmative – How are we still doing?. Formative Evaluation. Even excellent plans and concepts can be improved Gauge success by classroom program presentation

dillian
Download Presentation

Developing Evaluation Instruments

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Developing Evaluation Instruments

  2. Evaluations • Formative – How are we doing? • Summative – How did we do? • Confirmative – How are we still doing?

  3. Formative Evaluation • Even excellent plans and concepts can be improved • Gauge success by classroom program presentation • Informs the instructor how well instructional package is serving the objectives as it progresses

  4. Formative Evaluation • Program of instruction is evaluated through • Test results • Learner reactions/comments • General observation of learners during class • Subject-matter expert reviews • Colleague suggestions • Quality control of the development process

  5. Formative Evaluation • Questions that may be asked • Level of learning acceptable compared to objectives? • Learners using knowledge to perform skills? • Time element acceptable? • Activities appropriate? • Tests measuring objectives?

  6. Summative Evaluation • Measures success of major outcomes • Usually follows all instruction, projects and testing in a program

  7. Summative Evaluation • Measures • Learning efficiency (material mastered/time) • Program cost (Development and delivery) • Continuing expenses • Student comments/reactions/evaluations • Long-term benefit of program

  8. Confirmative Evaluation • Although instruction initially effective, some problems appear over time • Tracks learner experiences over time to judge program’s validity • Relies upon numerous data-collection instruments (interviews, performance assessments, etc.)

  9. Confirmative Evaluation • Questions to investigate • Do learners continue to perform correctly over time? • Do materials still meet original objectives? • How has technology and/or resources, trends, and attitudes changed since inception? • How to best meet clients’ needs over time?

  10. Confirmative Evaluation • Changes needed in materials? What are the costs? • If instruction not working as well as before: • Should instruction continue as is? • Should it be revised? • Should it be terminated? • What might replace it?

  11. Validity • The test assesses what it is supposed to measure – For example: • Performance tests assess processes and outcomes relating to skills or competencies • Course attitude surveys need to measure reactions to the course

  12. Reliability • The test will produce consistent results whenever used • More questions relating to specific objectives = more reliable test • Standardized delivery of test • Scoring methods – the less subjective, the better

  13. Pretesting • Judges the learner’s preparation to study the material (course or topic) • Determines competencies already mastered

  14. Testing for Prerequisites • Standardized paper-and-pencil test • Performance observations • Questionnaire (Looks at learner background, training, and experiences) • Review of past work experience • Interview past supervisors/managers

  15. Objective tests • Multiple choice • True or False? • Matching items

  16. Constructed-Response • Make learner plan answers and express them in their own words • Short-answer • Essay questions • Problem solving

  17. Skills and Behavior • Actions observed • Process, product, or both? • Constraints/limitations? • Testing conditions simulated or realistic?

  18. The Rubric • General assessment of overall product • Certain elements attain points • More objective tool From Designing Effective Instruction 4th Ed., Morrison, Ross, & Kemp, 2004.

More Related