Assessment Instruments: Assessing learners

1. Instructional DesignEvaluation, Assessment & Design: A Discussion (EDER 673 L.91 )From Calgary With Asst. Professor Eugene G. Kowchmarch 27(An Synchronous Meeting using WebCT discussion Thread and WWW Course Home Page Material) iD

2. Assessment Instruments: Assessing learners • Criterion Referenced • Pretest: Placement & entry level skills • Readiness Questions • Placement Questions • Posttest: helps us identify what did not work for this learner • --------------------- • To make these tests, we need a lot of info: • List of task based behavior objectives or • Type of learning (verbal, etc, ) and • Type of conditions for learning (Gagne) and events for learning • Bad criteria test subjects: • Writing a paragraph…affective evaluations….. iD

3. Pretest: Method of Inquiry for Placement From Driscoll, M. (2000). Psychology for Instruction, 2nd Ed. Placement Learner Readiness Assessment (to determine entry performance) Readiness ConcernDo the students have the Prerequisite skills? Placement Concern Have the students already Achieved the intended Outcomes? yes no yes no Provide readiness experiences Proceed with The instruction Advance Students To a higher level Discuss: Math vs. Social Studies (criterion?) iD

4. Formative and Summative Assessments:( of your Design) • Formative methodology (can be done in order): • Education Context: 1 to 1 evaluation (designer & learner) • Questionnaires, interviews - check learning time, outcomes • Focus group • Nets out the instructor • Check the effectiveness of instruction (viability, attitudes of instr.) • Field Trial • Observe a full instruction event with the 2x modified course materials and design… test for achievement. iD

5. Formative and Summative Assessments:( of your Design) • Summative Assessment methodology • Expert Judgment (assess the instructor) • Congruence; Can the instructor meet the needs of the organization? • Content: Are the materials complete, updated? • Instructor followed the design? • Utility of instructor? • Design: Are the principles of learning, instruction and motivation in the materials? Instruction? • Field Trial • Outcomes Analysis: impact to learner on job, in school • Management: Instructor, supervisor performance test iD

6. Summative Learner Assmt. Method Summative Assessment (to monitor learning progress) Have the students achieved The intended outcomes Of instruction? no yes Provide additional learning experiences Assign grades to certify mastery Evaluate the effectiveness of the instruction Discuss iD

7. Assessment Planning (Magliaro from notes) • Assessment: Learner performance • Evaluation: judging learner decisions • Program Evaluation: Appraising program success • Assessment Possibilities • Portfolios • Physical • Electronic • Communicate task and knowledge performance • Time consuming for all! • Projects • Demonstrate action / processes / competencies (training) • Address problem solving (education) • Require relevant subject matter! iD

8. Assessment Planning (Magliaro from notes) • 5 Step assessment process • Identify goals and objectives from Needs Assmt • Match goals & objectives with assmt tools • Choose instructional method • Design tasks and scoring methods (match) • Think through the implications…. Guidelines: • Start small • Develop rubrics • Communicate purpose of assessment • Involve participants • Keep going - recursive design • Look for blips - unfairness, odd results iD

9. PostTest: Method of Inquiry for Placement The instructional role of FormativeAssessment During Instruction (to monitor learning progress) Are the students achieving the intended Learning outcomes? no yes Provide Group or Individual remediation Provide feedback to reinforce learning Proceed with The planned instruction DIAGNOSTIC ASSESSMENT (to study persistent difficulties iD Discuss

10. Recall: The “one” and the “many” iD

11. Norm The Distribution Curve: Defining Normal Relative ranking, describes what cannot be done And what can be done.. iD

12. Designers Ask.. Great questions: "Can and should instructional designers place importance on process instead of product? If yes, is this desirable and reversible? Discuss" iD

13. Designers Ask.. Great questions: Should and can evaluation be both internal and external? Should and can the evaluator personalize his or her findings and put those results into a cultural perspective? Discuss." iD

14. Designers Ask.. 1) There is the sense that "outside the company" designers are not as biased as "backyard" designers. I wonder whether this is in fact true, or whether their biases are simply different. The "outside" designers could have biases that are harmful or just as blinding as "in house" people's. iD

15. Designers Ask.. Evaluation/Assessment There seems to be a belief that objectivity is somehow better than subjectivity, and I wonder about that too. Sometimes, a subjective approach may be called for. For example, in evaluating a Native education program, would it be effective to use a white person? Maybe. Then again, maybe not. Maybe the white person would have to spend so much time and effort learning first about the culture that her work on actual evaluation would become miniscule in comparison. iD

16. iD