1 / 61

Assessing Distance Learning: Meeting Higher Expectations for Online Learners

This article explores the importance of carefully designing assessments for online learners and evaluates the effectiveness of distance education programs in meeting student learning outcomes. It also discusses the assessment methods used at Carteret Community College.

dgorby
Download Presentation

Assessing Distance Learning: Meeting Higher Expectations for Online Learners

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. “When learners are at a distance, the careful design of assessments is particularly important, because society somewhat unfairly imposes higher expectations on the assessment of online learners.” Oosterhof, et al. (2008) Assessing Learners Online

  2. Assessing Distance Learning: Are they learning what we think we’re teaching? ? ? ? ? ? ? Donald Staub, Title III Director Carteret Community College

  3. http://ccctitle3.wordpress.com/

  4. The Lineup • DL @ CCC • Why Worry about DL Assessment? • How We’re Doing It • The DL Program • DL in Programs & Courses • Possibilities & Challenges

  5. DL @ CCC • 33 programs @ CCC • FA10 enrollment ~1800 • DL is 10+ years old • FA10 • 336 total sections • 243 seated sections (72%) • 64 Internet (20%) • 29 Hybrid (8%) • 58% FT / 42% PT 5

  6. Why worry about assessing DL? @ the College & Program-Level: • Student Success • Persistence • (btw: success vs. retention vs. persistence) • “Good assessment guides good PD/ID”

  7. @ the Course-Level: Generalizability (c.f. Oosterhof, et al, 2008) “…the degree to which learner performances that are observed generalize to performances that are not observed.” [i.e. are they learning what we think we’re teaching?]

  8. So, how are we doing this?Caveat: We’re not saying this is right or wrong…it’s just the way that we’re currently approaching it.

  9. “Comparability of distance education programs to campus-based programs and courses is ensured by the evaluation of educational effectiveness, including assessments of student learning outcomes, student retention, and student satisfaction.” (CS 3.3.1) COC Distance Education Policy Statement http://www.sacscoc.org/pdf/Distance%20and%20correspondence%20policy%20final.pdf

  10. The (former) Guidelines

  11. Q1: Is the distance education program reflected in the institution’s strategic (or master) plan? 2009-2012 Strategic Plan We will create a culture where all institutional actions are focused on improving student learning and success and leading to student goal completion. PM 16—Success and Withdrawal Rates by Instructional modalities

  12. Q2: Is there evidence that outcomes for the program have been identified? DL-AO #1: Faculty demonstrate the ability to teach in the online environment in a proficient (student centered) manner. DL-AO #2: Students have more educational options to pursue degrees, diplomas, and certificates due to expanded online course offerings at CCC.

  13. Q3: Is there evidence that the effectiveness of the distance education program is regularly assessed and steps taken for improvement of the program? AO #1: Faculty demonstrate the ability to teach in the online environment in a proficient (student centered) manner. Assessment: Quality Assessment Plan, Success/ Persistence Rates, Student Surveys/Evaluations. AO #2: Students have more educational options to pursue degrees, diplomas, and certificates due to expanded online course offerings at CCC. Assessment: The number of internet/hybrid sections offered.

  14. QAP

  15. The QAP(Quality Assessment Plan) • Developed in-house (using models) • Peer-reviewed by content specialist • Stipends (T3) • All current online courses have been reviewed • All new online courses must be reviewed/ certified • http://web.carteret.edu/keoughp/TitleIII/FinalQAP.xls

  16. The QAP Score Card

  17. The QAP Score Card: Learning

  18. QAPSuccess/Persistence

  19. Success - Spring 2010 • 75 sections • seated: 75% success (persist: 87.1%) • internet: 66% success (persist: 81.7%) • hybrid: 73% success (persist: 87.8%) • FA09 internet: 62 sections; 68% success • FA08 internet: 61 sections; 67% success • FA07 internet: 59 sections; 66% success • FA06 internet: 48 sections; 66% success 20

  20. QAPSuccess/PersistenceProgram/Course Evaluations

  21. Surveys & Evaluations: DL Program

  22. Surveys & Evaluations: DL Program 23

  23. Surveys & Evaluations: Course-Level

  24. Q3: Is there evidence that the effectiveness of the distance education program is regularly assessed and steps taken for improvement of the program? • Modality Comparison • Grade Distributions • Program Reviews • ILLOs (Gen Ed outcomes)

  25. Modality Comparison

  26. Modality Comparison Institutional-level

  27. Modality Comparison Program-level

  28. Modality Comparison Course-level

  29. Modality ComparisonGrade Distributions

  30. Grade Distributions institution-level %s

  31. Grade Distributions course-level

  32. Modality ComparisonGrade DistributionsProgram Reviews

  33. Evaluate Instructional Modalities

  34. Modality ComparisonGrade DistributionsProgram ReviewsILLOs

  35. ILLOs (Gen Ed outcomes) • Communication • Computer Literacy • Critical Thinking • Humanities & Fine Arts • Information Literacy • Personal Growth & Responsibility

  36. Computer Literacy - CIS 110 Outcome: Students demonstrate proficiency in the use of computer technology including application software, system software, and internet browsers. Assessment: Final Exam in CIS 110.

  37. Computer Literacy - CIS 110 Section analysis

  38. Computer Literacy - CIS 110 Same instructor teaching one section of each

  39. Personal Growth & Responsibility ACA 115 Outcome: Students demonstrate the ability to understand and manage self effectively. Assessment: Advisement assignment in ACA 115 & 118 evaluated by rubric. Results Spring 2009: 131 total projects submitted; 81% scored 80 or above. Seated 115: 85% scored 80 or above Online 115: 81% scored 80 or above

  40. Q4: Has the institution developed student learning competencies for the courses/programs offered by distance education? If these are the same competencies for courses/programs offered by “traditional” methodologies, is assessment identified for distance learning students separate from students taking courses by “traditional” methodologies?

  41. “Comparability of distance education programs to campus-based programs and courses is ensured by the evaluation of educational effectiveness, including assessments of student learning outcomes, student retention, and student satisfaction.” So, are they learning what we think we’re teaching?

  42. And the answer is… • Yes! … No?…maybe?? • PLLOs • Course-Level Assessment

  43. PLLOs

  44. PLLOs (SLOs) Same instructor teaching one section of each (common assignment from each section) • Business Administration • ACC 120 • BUS 110 • ECO 251/252

  45. PLLO: preparation of financial statements as part of mid-termAssessed in: ACC 120 47

  46. BUS 110 ECO 250

  47. PLLOsCourse-Level Assessment

  48. Course-level assessment - CLLOs • ACC 120 • ART 111 • EDU 119 • ENG 111 • HIS 111 • PSY 150 & 241 • RCP 114 • SOC 210 50

More Related