Assessing Online Instruction at Carteret Community College

1. “When learners are at a distance, the careful design of assessments is particularly important, because society somewhat unfairly imposes higher expectations on the assessment of online learners.” Oosterhof, et al. (2008) Assessing Learners Online

2. Assessing Online Instruction at Carteret Community College ? ? ? ? ? ? Donald Staub, Title III Director Carteret Community College

3. The Lineup • DL at CCC • The Need to Assess • How We’re Doing It • Conclusions & Solutions • Discussion

4. DL @ CCC • DL is 10 years old • 34 programs @ CCC • ~1700 enrollment SP09 • Fall 2008 • 348 total sections taught • 270 seated sections • 61 Internet • 15 Hybrid

5. The Need to Assess • SACS • Common Sense • it’s the right thing to do • we need some answers

6. “Comparability of distance education programs to campus-based programs and courses is ensured by the evaluation of educational effectiveness, including assessments of student learning outcomes, student retention, and student satisfaction.” • (CS 3.3.1) • COC Distance Education Policy Statement • http://www.sacscoc.org/pdf/081705/distance education.pdf

7. DL @ CCC • 270 seated sections • 73% success / 22% failed or withdrew (15% W) • 61 internet sections • 65% success / 33% D, F, WD (16% W) • 15 hybrid sections • 81% success / 18% D, F, WD (9% W) • FA07 internet: 59 sections; 34% D, F, W • FA06 internet: 48 sections; 34% D, F, W

8. How We’re Doing It • QAP • Grade Distributions • ILLOs & PLLOs • Same instructor

9. The QAP(Quality Assessment Plan) • Developed in-house (using models) • Peer-reviewed by content specialist • Stipends • All current online courses have been reviewed • All new online courses must be reviewed http://ccctitle3.wordpress.com/

10. The QAP Score Card http://ccctitle3.wordpress.com/

11. The QAP Score Card: Learning

12. Grade Distributions (Fall 2008)

13. ILLOs • Communication • Computer Literacy • Critical Thinking • Humanities & Fine Arts • Information Literacy • Personal Growth & Responsibility

14. Computer Literacy - CIS 110 Outcome: Students demonstrate proficiency in the use of computer technology including application software, system software, and internet browsers. Assessment:Final Exam in CIS 110. This exam consists of a hands-on skills assignment for Word, Excel, Access and PowerPoint and a 25 question Multiple choice assignment with questions regarding computer hardware, system software, internet browsers and other related computer terminology.

15. Computer Literacy • 5 Sections: CIS 110 3-WB; 2-IN • Attrition: • IN01: 35% • IN02: 26% • WB03: 17% • WB04: 28% • WB05N: 18% • Cross-section comparison (i.e. one course, different instructors)

18. Computer Literacy - CIS 110 Same instructor teaching one section of each

19. Personal Growth & Responsibility ACA 115 Outcome: Students demonstrate the ability to understand and manage self effectively.

20. Personal Growth & Responsibility ACA 115

21. Personal Growth & Responsibility ACA 115 • Advisement Project • In the fall of 2008, 79% of all ACA students (10 sections) scored 80 or above. • For online sections (2 sections), 70% (7 out of 10) scored 80 or above.

22. Personal Growth & Responsibility PSY 150

23. PLLOs Same instructor teaching one section of each (take a look at a common assignment from each section) • ACA 115 • ACC 120 • BUS 110 • ECO 251 • EDU 119

24. ACA 115 - Learning Styles Project Students identify and explain the strategy used to prepare for an exam.

25. ACC 120

26. BUS 110 ECO 250

27. EDU 119 4 test test items related to: working/communicating with parents, parental rights, and regulations dealing with parental contact/information. n=20 n=12

28. Conclusions & Solutions “The largest issue confronting online assessments is that of the dynamics that occur when the instructor and student do not share the same immediate space.” Milam, et al. (2004). In, Developing and Implementing Assessment of Student Learning Outcomes

29. Instructors who don’t treat online & seated equitably (e.g. additional support to seated sections) • Using the observation instrument correctly. Some of this was a direct result of them using a document that was scanned and they had to print/copy to use. I believe some of the students had trouble with the process. (EDU 119) • Math & the fax machine

30. “The seated students did however have the opportunity to submit their observation checks sheets for revision if needed.” “Students will no longer have to print the inventory results. Instead, they will write a narrative explaining their results.” • “Students will no longer need to submit a curriculum check sheet (on-line students had difficulty accessing them from the website).”

31. Discussion “In many instances, the assessment of an online course is not in alignment with the type of instruction that may have occurred in the course.” Palloff & Pratt (2009) Assessing the Online Learner

32. Donald Staub Title III DirectorCarteret Community Collegestaubd@carteret.edu 252- 222-6010 http://ccctitle3.wordpress.com/