Ishmael B. Said Macomb Community College 2013 MIAIR Conference

1. Online v. On-ground Courses: Drop-out Rates Analysis Implications for Online Course Management Ishmael B. Said Macomb Community College 2013 MIAIR Conference

2. Goals of Presentation Compare online and on-ground course sections with respect to two drop-out rates measures Show results Define, and interpret the new conceptualized drop-out rate (early drop-out) Discuss implication for online course management Identify a possible intervention aimed at managing the early drop-out rate and increasing retention Discussion and Questions

3. Introduction – Reporting Course Drop-out Rates • Drop-Out Rates Operationalization • Conventional Calculation (System generated) • DR= Drop-out (D) x 100 Registered (R) + Add (A) + Drop-out (D) • Early Drop-out Rate (Recalculated) • EDR = Drop-out (D) - Withdrawal (W) x 100 Registered (R) + Add (A) - Withdrawal (W) = Grade of W as a consequence of dropping a course ( usually after refund period) - Drop-out rate (DR) accounts for all drop-outs: early and late drop-out combined - End of semester count and level of aggregation (course vs. student level)

4. Results • Whole Sample Mean Differences • Drop-out Rate • Online: 27.2 (N= 275) • On-Ground: 18.5 (N= 748) (ΔMean = 8.7, p < .0001*) • Early Drop-out Rate • Online: 25.3 • On-Ground: 10.1 (Δ Mean = 15.2, p < .00001*) F ratio: Analysis of Variance

5. Results (Continued) • Selected Disciplines Mean Differences • Drop-out Rate • Business HumanitiesITMath • Online: 29.4 26.8 27.9 39.9 • On-Ground: 19.9 18.3 22.7 28.2 ΔMean* = 9.5 8.5 5.2 11.7 • Early Drop-out Rate • Online: 28.5 28.2 25.5 40.5 • On-Ground: 11.3 10.3 14.9 11.9 Δ Mean* = 17.2 17.9 10.6 28.6 *All mean differences significant at p< .001, F ratio

6. Results (Continued) • Course Length & Course Level Mean Differences • Drop-out Rate Length Level . • 16 weeks 8-12 weeks100200 • Online: 18.6 25.3 27.9 25.1 • On-Ground: 15.8 17.8 18.8 16.2 ΔMean = 2.8 7.5* 9.1* 8.9* • Early Drop-out Rate • Online: 10.1 23.2 25.8 24.0 • On-Ground: 8.2 10.4 10.2 9.3 Δ Mean = 1.9 12.8* 15.6* 14.7* * Mean differences significant at p< .001, F ratio

7. Conclusion • Online courses have a significantly higher overall drop-out rate than on-ground courses • Significantly larger difference between overall drop-out rate and early drop-out rate • Online courses account for more then twice the incidence of early drop-out when compared to on-ground courses • No significant differences when controlling for type of course, course duration and course level

8. Policy Implications • 1. Academic • Student’s willingness to sustain rigorous self disciplined learning attitude required for online courses • Readiness of students for demanding online course requirements • Academic aptitude interaction • Research needed

9. Policy Implications • 2.Institutional: Online course management • Full course refund payment (1st day of class) - Work overload • Added burden to regular on-ground course refund payment in addition to 1st week busy registration and enrollment period • Possible communication back-log associated with students’ waitlist management process

10. Helping Online Course Managers • Method: Boxplot (box-and-whiskers plot) • Visually powerful technique for detecting outliers (courses with higher than normal drop-out rates) • Comparing the two drop-out rates • Early drop out rate interpretation

11. Example: Drop-out Rate

12. Example: Early Drop-out Rate

13. Interpretation • Drop-out rate (DR) • Proportional to all students starting a course or having a transaction with the course • A course of 30% drop-out rate • 3 out 10 who started the course, dropped or withdrew from the course • No difference between early-drop out and later withdraws

14. Interpretation • Early drop-out rate (EDR) • Proportional to student remaining in the course • An example of 200% early drop-out rate (extreme case) • The number of students who drop early is as twice as large as those who remain in the course. • As an example, if 8 students remained, 16 students dropped the course during the early period. • End of term indicator of attrition not due to student academic performance • Readiness for online course requirements

15. EDR Examples: Academic Disciplines

16. EDR Examples: Social Sciences

17. Applied Research Agenda for Management of Online courses Intervention and Evaluation Research- Pilot study Possible campus wide implementation of intervention

18. Intervention • Open access to online course material one week prior to official start date of the course • Rationale • Allowing students to review course material for a subjective assessment of their abilities to meet the high demand of online course requirements • Hypothesis • Intervention will stretch the incidence of drop-out through the entire week instead of during the 1st day thus decreasing course early drop-out rate

19. Research Proposal • Purpose • Determine if intervention has the intended effect of decreasing the early drop-out rate • Methodology 1 • Quasi-Experimental Design • Posttest Comparison Group Design for causal link • 2 Groups: Opened access sections vs. non-opened access sections • Depending on sampling size • Effect breakdown by discipline (Post Hoc Multiple Comparisons)

20. Research Proposal for a Pilot Study • Methodological pre-requisites • Target Population: Online course sections • Stratified random sampling of sections by discipline and course level • Matching on all important variables • PCGD requires equivalency on major confounding variables • Open access sections vs. non-opened sections

21. Suggestions & Discussion • Applied research for decision making and institutional effectiveness • Are there other possible interventions that may help online managers decrease the incidence of early course attrition? • Are there other statistical models that would allow for a better estimation of the selected intervention? • Methodological issues with reliable research design • Consent • Participation • Monitoring

22. Ishmael B. Said Macomb Community College saidi@macomb.edu