Learner Assessment in an Online Environment

1. Learner Assessment in an Online Environment Patricia McGee, Alisa Cooper, Jennifer Freeman

2. Which best describes your experience with online assessments? • Developed my own assessments • Used publisher-created assessments • Worked with others to create assessments • Used assessments developed by someone other than publisher

3. Overview of Workshop • Classroom vs. Online Assessment • Best Fit Assessments for Online Courses • Designing with the End in Mind • Giving Assessment Directions &Feedback • Rubrics and Checklists • Involving the Learner

4. Assessment Toolkit P. McGee

5. Classroom vs. Online Assessment Patricia McGee

6. Instructional Functions of Assessment

7. One to many • Just in time assessment • Immediacy • Focus on methods • One to one • Just in need assessment • Ambiguity • Focus on strategies

8. Differences? • Time • Security • Integrity • Resources • Learner autonomy • Lack of just in need responses • Getting, keeping, and directing attention • Clarity of instructions and expectations • Other?

9. Challenges? Cheating Communication Detailed expectations, e.g., rubrics Respond within a pre-designated timeframe Peer critique review before final submission Benchmarks, points of reflection, accomplishment & completion • Randomized objective quizzes/tests • Timed and authenticated • Multiple ways to demonstrate understanding • Choices • Duplication • Repetition • Revise and resubmit • Small group assessments for accountability

10. FERPA?

11. Activity • Brainstorm for assessment challenges. • Discussion possible solutions • Be prepared to share a particularly difficult challenge.

12. Assessment Toolkit P. McGee

13. Progressive Assessment & Benchmarks Patricia McGee

15. Learners should know…. • That an assessment is coming • A word about helping students plan • What happens after the assessment • How to prepare for an assessment • What participating in the assessment will look like • Practice

16. Classroom (online) Assessment Techniques • Informal • Ungraded • Low threat and no risk • Inform instructor about student learning • Inform learner about learning progress From Classroom Assessment Techniques

17. CATS Examples: Prior Knowledge CAT Online Application Survey or Poll Survey Monkey™ Poll Everywhere™ PollDaddy™ Background Knowledge Probe: short, simple questionnaires prepared by instructors for use at the beginning of a course or at the start of new units or topics; can serve as a pretest

18. CATS Examples: Prior Knowledge CAT Online Application Shared Notetaking/Concept Mapping Bubblus™ Mind42™ Mindomo™ Wordle™ Empty Outlines: in a limited amount of time students complete an empty or partially completed outline of an in-class presentation or homework assignment

19. 5 Suggestions for Online CATs • Customize to specific needs and learning environment • Be consistent with the instructional philosophy • Test out a CAT and assess effectiveness • Allow extra time to carry out and respond to the assessment • Let students know what you learn from their feedback and how you and they can use that information to improve learning

20. ACTIVITY • Review the CATs and pick one • Identify the goal for your CAT • Explain why this CAT is helpful/necessary in this particular area of the course • How and when will students receive feedback on the CAT?

21. Progressive Assessment

22. Online Progressive Assessment Interaction Activities Assignments Participation

23. Performance vs Objective Assessments Objective Performance Activity-Based Cases Projects Debates/negotiations Production Field experience Products Presentations Research paper • Classroom Assessment Techniques • Multiple choice tests • Embedded Test Questions • Pre and Post Tests • Discussion Threads

24. Example: Periodic Assessments

25. Example: benchmarks

26. Example: progressive benchmarks

27. Example: Assignment BENCHMARK?

29. Activity: Progressive Assessments • What are most comprehensive benchmarks for a course? • What are challenges? Issues?

30. Design with the end in mind Alisa Cooper

31. Provide multiple and alternative modes of assessments

32. Learning and Assessment Cycle

34. Bloom’s Taxonomy Backwards Design Focus on learner Focus on measure of learning

35. Assessment & Bloom’s

36. ACTIVITY • Using handout, determine at what level of Bloom’s you are assessing • Explore tools that might allow you to assess at higher levels of Bloom’s see http://elearningtools.wetpaint.com

37. Giving Assessment Directions & Feedback Jenni Freeman

38. Assessment Instructions This seems like a fundamental concept, but many assessments lack adequate instructions, leading students to answer questions incorrectly or without the desired depth. Two levels: • How to use the tools, logistics • Student expectations

39. Assessment Instructions: Questions to Answer • Objective Tests • Using the tool? • How many times available? • When available? • Results available? • Other?

40. Assessment Instructions: Questions to Answer • Subjective Assessments • How to submit? • When due? • How will it be graded? What does a successful paper look like? • Results available? • Other?

41. Assessment Instructions: Best Practices • Separate general instructions (How to use the tools, logistics) from assessment-specific instructions (due dates, rubrics) • Centrally located pop-up • Syllabus • Designated discussion forum • Provide grading rubrics • Provide models if appropriate (an “A” paper and an “F” paper)

42. Assessment Feedback • The sooner the better • Instant results (objective tests) • Auto-reply upon receipt (subjective assessments) • Details • Feedback for all distracters, not just the wrong answers (objective - students who guess correctly will also receive benefit) • Compose boilerplate language and phrases for common comments to students (subjective)

43. Assessment Feedback • The sooner the better • Instant results (objective tests) • Auto-reply upon receipt (subjective assessments) • Details • Feedback for all distracters, not just the wrong answers (objective - students who guess correctly will also receive benefit) • Compose boilerplate language and phrases for common comments to students (subjective)

44. ACTIVITY • Write instructions for your sample assessment keeping all of the above in mind. Share with your table. • What sort of feedback will your students need after completing the assessment?

46. Frequency of Feedback Pedagogical Timing Calendar Schedule Indirect – weekly Direct - bi-monthly • After practice • At completion/achievement of objective

47. For example Chat – content learning Chat - performance Participate throughout discussion Do not interrupt Respond when called upon When appropriate, come to chat prepared. • Answer questions correctly • Ask questions that • Helped others learn • Extended discussion • Make contributions to discussion that extended materials

48. Outcome + Criteria = Assessment • How are you communicating that to learners? • How will you document the learning ? • Do you expect learner to learner a new tool for assessment ? • Have you provided for practice before assessment?

49. Using Rubrics • Allows assessment to be more objective and consistent • Focuses instruction to clarify criteria in specific terms • Clearly shows the student how their work will be evaluated and what is expected

50. Rubric Tips • Ensure that the assessment criteria can be directly correlated to the course objectives. • Break the criteria down as specifically as possible and clearly define exemplary, satisfactory, and unsatisfactory efforts for each. • Be sure that students have access to the rubric before they begin the assignment.