Completing Learning Support: Using Predictive Modeling to Determine Best Practices

1. Completing Learning Support: Using Predictive Modeling to Determine Best Practices Gregory SchutzRion McDonald Chris Tingle TASSR Fall ConferenceOctober 24, 2013

2. Successful academic entry is a necessarycomponent for degree attainment. • The Complete College Tennessee Act mandated that public higher education institutions in Tennessee improve student success. • Meanwhile, approximately two-thirds of incoming freshmen at Tennessee Board of Regents institutions required some sort of pre-college learning support in fall 2011.

3. Student abilities in math, reading, and writing skills impact successful entry. • The TBR adopted new learning support (LS-precollege) guidelines with the goal that all students be able to move through LS in a timely manner. • Holding demographic and academic preparedness constant, the authors find one-year, LS completion rates differ by institution.

4. Presentation Flow • Purpose • Background • Findings • Recommendations

5. What Matters? • Introduce evaluation data being collected a the Tennessee Board of Regents (TBR) for their Learning Support (developmental education) initiative. • Illustrate a predictive model that includes academic preparation, demographics, and institutional components for completing LS. • Provide a method and tools for researching the impact of Learning Support (developmental education) • Discuss implications on best implementation practices and future research.

6. For this Study Completion of LS is defined as: Completion of all learning support competencies within the first semester or first year of enrollment.

7. Outcomes for Learning Support Math * No significant difference

8. Outcomes for Learning Support Writing * No significant difference

9. Outcomes for Learning Support Reading * No significant difference

10. Literature Review: Student Background • Part-time students are significantly less likely to complete the learning support program. • Adult students and under-represented minority students are both more likely to need remediation and also less likely to complete all required learning support. • The lower the high school grade point average, the less likely a student is to complete all required learning support. • Students that are required to take more learning support are less likely to complete the learning support curriculum.

11. Literature Review: Best Practices • Mainstreaming learning support students into college level courses while providing supplemental instruction. • Immediate credit accumulation, increased retention. • May not work for high need students. • Condensing semester long learning support courses into shorter timeframes and stacking courses for quickest completion. • Reduced withdrawals from learning support courses. • Modularizing learning support competencies into specific skills in order to reduce a student’s needed learning support. • Increased success of first college level course and increased retention. • Increasing student support, especially additional advising or tutoring, positively affects learning support outcomes.

12. Learning Support Guidelines • All students must meet ACT college benchmarks or be diagnosed for and placed into appropriate learning support. • Institutions will design learning support curriculum so that full-time students can satisfy pre-college level requirements in one semester. • Institutions must structure learning support so that a student who has demonstrated mastery of a competency will not be required to repeat support in that area. • Delivery of learning support must be based on proven methods of integrating technology as a tool for instruction. • Universities will not award credit that is less than college level.

13. Learning Support Study http://www.highereducation.org/crosstalk/ct0105/news0105-virginia.shtml (retrieved 8/3/13)

14. Logistic Regression Model Log (Odds of Completion)= constant + coefficient(ACT Score) + coefficient(HS GPA) + coefficient(# Competencies Required) + coefficient(Attendance Status) + coefficient(Age) + coefficient(Minority Status) + coefficient(School1) + coefficient(School2) + coefficent(School3) +….etc. + error Probability of Completion Odds of Completion = (1 –Probability of Completion)

15. Math Model: Student-Related Factors * ceteris paribus

16. Math Model: School Effects * deviation from average, ceteris paribus

17. Math Model: Predicted Probabilities * constant student-related factors used across schools

18. Writing Model: Student-Related Factors * ceteris paribus

19. Writing Model: School Effects * deviation from average, ceteris paribus

20. Writing Model: Predicted Probabilities * constant student-related factors used across schools

21. Reading Model: Student-Related Factors * ceteris paribus

22. Reading Model: School Effects * deviation from average, ceteris paribus

23. Reading Model: Predicted Probabilities * constant student-related factors used across schools

24. The data implies a set of recommendations: • The ultimate aim of graduation and the near future goal of completing entry level math, writing and reading courses need to be evaluated further to confirm the strength of completing learning support as a leading indicator. • Continue to follow if one-semester completion rates impact college success more strongly than one-year completion rates.

25. The literature implies another set of recommendations: • Continue to investigate mainstreaming learning support students into college level courses while studying where the level of unpreparedness may make the method unfeasible. • Look more closely at age to see how age is impacting learning support completion. • Pilot services like learning support to first-time college students just meeting TBR college readiness goals. • Target LS delivery and services for students of different backgrounds, demographics, and majors.

26. Institution Best Practices • Institution should commit to finishing in one semester. • Student and instructor interactions. • Facilitate LS competency completion as a prerequisite or corequisite for college level courses. • Create a data driven decision-making culture. • Coordinate LS functions on campus. • Maintain a faculty-led classroom while implementing new technology. • Require professional development for full-time and part-time faculty. • Promote use of full-time faculty in the classroom.

27. Conclusion • TBR has changed policy and practice to meet literature recommendations on college readiness and initial indications show improved results for institutions implementing these policies. • The completion of learning support in a timely fashion is a leading indicator for underprepared first-time college students. • The variation of success in completing learning support across campuses suggests that best practice institutions can be identified by results. • The success of learning support with underprepared students suggests that the process of identifying and delivering competency levels could be piloted for other university and college populations. • TBR can use system level data sources to help campuses with their 2015-2020 strategic planning and for evaluating the possible impact on college completion (trajectories).

28. Completing Learning Support: Using Predictive Modeling to Determine Best Practices Gregory Schutz – Greg.Schutz@tbr.eduRion McDonald – RMcDonald11@columbiastate.edu Chris Tingle – Chris.Tingle@tbr.edu TASSR Fall ConferenceOctober 24, 2013