610 likes | 762 Views
“When learners are at a distance, the careful design of assessments is particularly important, because society somewhat unfairly imposes higher expectations on the assessment of online learners.” Oosterhof, et al. (2008) Assessing Learners Online.
E N D
“When learners are at a distance, the careful design of assessments is particularly important, because society somewhat unfairly imposes higher expectations on the assessment of online learners.” Oosterhof, et al. (2008) Assessing Learners Online
Assessing Distance Learning: Are they learning what we think we’re teaching? ? ? ? ? ? ? Donald Staub, Title III Director Carteret Community College
The Lineup • DL @ CCC • Why Worry about DL Assessment? • How We’re Doing It • The DL Program • DL in Programs & Courses • Possibilities & Challenges
DL @ CCC • 33 programs @ CCC • Enrollment ~1800 • DL is 10+ years old • Spring 2011 • 399 total sections • 261 seated sections • 90 Internet (23%) • 40 Hybrid (10%) • 8 Web-based 5
Why worry about assessing DL? @ the College & Program-Level: • Student Success • Persistence • (btw: success vs. retention vs. persistence) • “Good assessment guides good PD/ID”
@ the Course-Level: Generalizability (c.f. Oosterhof, et al, 2008) “…the degree to which learner performances that are observed generalize to performances that are not observed.” [i.e. are they learning what we think we’re teaching?]
So, how are we doing this?Caveat: We’re not saying this is right or wrong…it’s just the way that we’re currently approaching it.
“Comparability of distance education programs to campus-based programs and courses is ensured by the evaluation of educational effectiveness, including assessments of student learning outcomes, student retention, and student satisfaction.” (CS 3.3.1) COC Distance Education Policy Statement http://www.sacscoc.org/pdf/Distance%20and%20correspondence%20policy%20final.pdf
Q1: Is the distance education program reflected in the institution’s strategic (or master) plan? 2009-2012 Strategic Plan We will create a culture where all institutional actions are focused on improving student learning and success and leading to student goal completion. PM 16—Success and Withdrawal Rates by Instructional modalities
Q2: Is there evidence that outcomes for the program have been identified? DL-AO #1: Faculty demonstrate the ability to teach in the online environment in a proficient (student centered) manner. DL-AO #2: Students have more educational options to pursue degrees, diplomas, and certificates due to expanded online course offerings at CCC.
Q3: Is there evidence that the effectiveness of the distance education program is regularly assessed and steps taken for improvement of the program? AO #1: Faculty demonstrate the ability to teach in the online environment in a proficient (student centered) manner. Assessment: Quality Assessment Plan, Success/ Persistence Rates, Student Surveys/Evaluations. AO #2: Students have more educational options to pursue degrees, diplomas, and certificates due to expanded online course offerings at CCC. Assessment: The number of internet/hybrid sections offered.
The QAP(Quality Assessment Plan) • Developed in-house (using models) • Peer-reviewed by content specialist • Stipends (T3) • All current online courses have been reviewed • All new online courses must be reviewed/ certified • http://web.carteret.edu/keoughp/TitleIII/FinalQAP.xls
Success - Spring 2010 • 75 sections • seated: 75% success (persist: 87.1%) • internet: 66% success (persist: 81.7%) • hybrid: 73% success (persist: 87.8%) • FA09 internet: 62 sections; 68% success • FA08 internet: 61 sections; 67% success • FA07 internet: 59 sections; 66% success • FA06 internet: 48 sections; 66% success 20
Q3: Is there evidence that the effectiveness of the distance education program is regularly assessed and steps taken for improvement of the program? • Modality Comparison • Grade Distributions • Program Reviews • ILLOs (Gen Ed outcomes)
Modality Comparison Institutional-level
Modality Comparison Program-level
Modality Comparison Course-level
ILLOs (Gen Ed outcomes) • Communication • Computer Literacy • Critical Thinking • Humanities & Fine Arts • Information Literacy • Personal Growth & Responsibility
Computer Literacy - CIS 110 Outcome: Students demonstrate proficiency in the use of computer technology including application software, system software, and internet browsers. Assessment: Final Exam in CIS 110.
Computer Literacy - CIS 110 Same instructor teaching one section of each
Q4: Has the institution developed student learning competencies for the courses/programs offered by distance education? If these are the same competencies for courses/programs offered by “traditional” methodologies, is assessment identified for distance learning students separate from students taking courses by “traditional” methodologies?
“Comparability of distance education programs to campus-based programs and courses is ensured by the evaluation of educational effectiveness, including assessments of student learning outcomes, student retention, and student satisfaction.” So, are they learning what we think we’re teaching?
And the answer is… • Yes! … No?…maybe?? • PLLOs • CLLOs
PLLOs (SLOs) Same instructor teaching one section of each (common assignment from each section) • Business Administration • ACC 120 • BUS 110 • ECO 251/252
PLLO: preparation of financial statements as part of mid-termAssessed in: ACC 120 45
BUS 110 ECO 250
Course-level assessment - CLLOs • ACC 120 • ART 111 • EDU 119 • ENG 111 • HIS 111 • PSY 150 & 241 • RCP 114 • SOC 210 48
Course-level assessment ~ CLLOs BUS 110 … Students will be able to: A: Explain Business Ethics and Social Responsibility B: Describe the major concepts of business, such as Entrepreneurship, Management, Marketing, Human Resource Management, and Financing C: Describe Business Globalization