If You Grade it Will They Learn? A Study About Assessment in Online Courses

1. If You Grade it Will They Learn? A Study About Assessment in Online Courses Bridget Arend University of Denver

2. Why is assessment important?

3. “Will this be on the test?”

4. Importance of assessment • Students typically concentrate their efforts on whatever content or cognitive skills they believe will be tested (Black & William, 1998; Bull & McKenna, 2004, Shepard, 2000) • Different forms of assessment encourage different types of learning (Gipps, 1994; Hynd et al., 2000; McKeachie et al., 1986; Pintrich, 1989) • The form of an exam or essay question can affect how students study (Entwistle, 1996) • The nature of online discussion questions influence the types of responses (Meyer, 2004)

5. Criticisms of HE Assessment • Classroom assessment often encourages superficial and rote learning (Black & William, 1998) • Grading is over-emphasized while learning is under-emphasized (Assessment Reform Group, 1999) • Higher education instructors do not give priority to providing feedback to students on their learning (Cross, 1988)

6. The Big Lesson From the Literature: Assessment for learning rather than assessment of learning

7. Are online assessment practices different?

8. Dissertation Study • Site = CCCOnline • Two phase, mixed-method design Phase I/Quantitative = 60 courses across disciplines • Surveys of 411 students and 51 instructors • 60 course observations Phase II/Qualitative = follow up of 9 courses • Email/phone interviews with 29 students and 8 instructors • 9 course observations

9. Assessment Practices Summative and Formative Assessment

10. Summative Assessment • Thomas Angelo’s framework of AAHE’s 9 Principles of Good Practice for Assessing Student Learning (1992) Principle #2 • Use multiple methods • Assess multiple dimensions • Use multiple assessors • Assess over time

11. Study Findings A “typical” online course: • 29 assignments using 5 different methods • Students use a variety of learning strategies • 65% use non-graded assignments, No peer/group work • Assignments due 10 out of 15 weeks

12. Formative Assessment • Non-graded assignments • What makes formative assessment successful? • Frequent and timely feedback • Precise feedback • Students using feedback • Changing course content or teaching methods when needed

13. Study Findings A “typical” course: • 7 non-graded assignments • Instructor interacts with each student at least 20 times, responds within 24-48 hours • Instructors give precise feedback 96% of the time • Instructors/students use feedback to improve course/learning only half of the time • Even with rubrics, students wonder why they got a certain grade

14. Positively related Discussions Written assignments (Problem assignments, Journals) Negatively Related Finals/Midterms Non-graded assignments (Knowledge-based, multiple-choice questions) Study Findings Critical Thinking Strategies

15. Study Highlights • Discussions • Play informal, formative role • Instructor role = neutral, participate infrequently, probe/challenge students

16. Study Highlights • Knowledge-Based, Multiple-Choice Exams • Least useful for understanding student knowledge • Instructors use as discipline tool or because they are supposed to • Has evolved from summative to formative role

17. Take-aways

18. Thank you! Bridget Arend University of Denver bridget.arend@du.edu http://portfolio.du.edu/barend