SAAIR Annual Conference Windhoek, Namibia 24-26 October 2017

1. Breaking-down the Silos: Working together towards solving the persistent problem of student success Marian Neale-Shutte, Qobo Qwaka, Andrea Watson & Kim Hurter SAAIR Annual Conference Windhoek, Namibia 24-26 October 2017

2. Outline Institutional Profile Law Faculty Profile Collaboration and Aims & Objectives Sample and Methodology Results Descriptive Analyses Results Logistic Regression Analyses Implications and Reflections “Education is the most powerful weapon which you can use to change the world.” - Nelson Mandela

3. NMU Institutional Profile

4. Nelson Mandela University Profile NMU Institutional Profile *Source: Office for Institutional Planning, Infographic Q1

5. Nelson Mandela University Faculty of Law Profile – WHO ARE OUR LAW STUDENTS? Nelson Mandela University Law Faculty Profile ACADEMIC FTE STAFF:STUDENT RATIO (2016) BY FACULTY SUCCESS RATES OF ALL COURSEWORK MODULES *Source: Office for Institutional Planning, Infographic Q2

6. Early warning, tracking & monitoring system • Under the auspices of the Siyaphumelela project. • Pilot institutional E,W,M&T system. • Draws in & integrates data from different databases. • More integrated, real-time picture of students academic performance. • Dashboard-like interface - will use indicators to flag students “in need of support.” • Inform: • The indicators through research. • Interpretation of student data and trends. • “Just-in-time” targeted interventions.

7. Working together Provide a broader view of students & student success. Tell a more complete Story. Develop more comprehensive plans for Interventions. Encourage collaboration & capacity building. OIP CAAR LAW • HEMIS data • Intelliweb data and reports • Findings from Institutional Research studies • Institutional and National Trends • CAAR data • Academic programme performance and interplay with biographical, school and Access Assessment variables • Module marks • Module performance, throughput, and anecdotal information DATA RICH

8. Aim & Objectives • Obtain a student description/profile. • Identify progression pathways & suggest benchmarks for risk identification. • Identify predictors of students at risk/ or “in need of support” CONNECT THE PARTS Investigate the relationship between pre-university demographic and education variables and academic success FORMATION OF A WHOLE

9. Sample & Methodology • Descriptive & pathway analyses: • Obtain a student description/profile. • Identify progression pathways & suggest benchmarks for risk identification. • Logistic Regression analyses: • Identify predictors of students at risk or “in need of support” for LLB/LLB Extended CONNECT THE PARTS • SAMPLE: n=725 • All 1st time entering, first year students registered in the Law Faculty from 2011-2015 • Registered for: • LLB • LLB EXT • Completed the CAAR access assessment FORMATION OF A WHOLE

10. Transforming data for the early warning, tracking & monitoring system WHY? WHAT’S BEST? VALUE HOW TO? WHAT? https://kvaes.wordpress.com/2013/05/31/data-knowledge-information-wisdom/

11. Student Profile - LLB (n = 497)

12. Student Profile - LLB Extended (n = 228)

13. Pathway Indicator of Sequential Choice [PISC] Main objectives of the PISC: • Understand the pathways that students follow through the LLB and LLB Extended programmes • For the current presentation • Identify the pathways followed • Identify points for targeted intervention • For further investigation • Identify student profiles for specific pathways followed • Identify predictors of academic performance for the different pathway categories *Robinson, R.A. (2004). Pathways to Completion: Patterns of Progression through a University Degree. Higher Education, 47, 1, pp. 1-20.

14. PISC – Graduated students Student progression to graduation occurred over a period of 4, 5 or 6 years with some stop-outs and transfers out of the faculty and then back into the LLB. 97 Graduated LLB/Ext LLLL_G LLLL_S_L_G LLLLL_G LLLLLL_G LTTTL_G XXXXX_G XXXXXX_G 7

15. PISC- Continuing students Continuing in LLB/Extended LL LLL LLLL LLLLL LLLLLL L_S_LL L_S_S_S_LL L_SL LL_S_LL LL_S_LLL LL_S_S_L XX XXX XXXX XXXXX XXXXXX X_S_X X_S_XXX Of the students still continuing we can immediately see where intervention, or at a minimum monitoring, might be important. Continuing in BA(Law) L_S_S_LB LLB LLBB LLBBB LLLBB XXXBB XXXXBB 97 18 25

16. PISC – Transferring Students Registered L_S_S_T L_S_T LLT_ST LLT_S LLLTT LLLTTT LT LLT LTT LLTTT LTTT LTTTT LTTTTT XXTT XTT XXXTT XT XXXT XXXTTT XXX_S_T XXTTT What are the pathways of students who start in the LLB or LLB Extended but end up somewhere else …? Graduated Other LL_S_TT_G LL_S_TTT_G LLTBB_G LLTT_G LLTTT_G LTTT_G LTTTT_G XTTT_G XXTTT_G Dropped-out LLLT_D LLT_D LT_D LTT_D LTTT_D XTT_D X_S_T_D XX_S_T_D XT_D XXTT_D 97 30 40

17. PISC – Departed Students Dropped-out L_D LL_D LLL_D LLLL_D LLLT_D LLT_D LT_D LTT_D LTTT_D X_D XTT_D X_S_T_D XXX_D XX_S_T_D XT_D XXXX_D XXTT_D XX_S_X_D XLL_D We tend to think of drop-out in simplistic terms, rather than in terms of persistence, time spent and resources spent before drop-out occurs … Stopped-out L_S LL_S LLL_S LLLL_S LT_S XT_S X_S XB_S XX_S XXX_S XXXX_S XXXTT_S XXXT_S XTT_S XXXXT_S 97 34 Lost to the university and potentially to law

18. Defining academic performance • Considered academic performance in three ways:

19. Determining patterns of academic performance • Looked at various patterns of academic performance across different kinds of groups of students. • The following groups of students performed differently: • Different school performance in terms of the cumulative APS • Had Maths as opposed to Maths Literacy • Had NSC English Home Language as opposed to another school Home Language • Were Directly Admitted (met minimum requirements) as opposed to those Tested Admitted (gained admission through access testing) • Males and Females • School Quintiles in terms of Quintiles 1-3 as opposed to Quintiles 4-5 and P/I

20. Logistic Regression - Variables LLB LLB Extended Dependent Variables: First Semester Academic Average Independent Variables: • Semester 1 modules: Introduction to Law (JLK111) Law of Persons (JLP111) Constitutional Law (JJT111) • Admission Point Score (APS) • Maths / Maths Lit • NSC Home Language • Gender • School Quintile • Tested Admitted / Directly Admitted Dependent Variables: First Semester Academic Average Independent Variables: • Semester 1 modules: Introduction to Law (JLK1X1) Law of Persons (JLP1X1) • APS • Maths / Maths Lit • NSC Home Language • Gender • School Quintile

21. Logistic Regression - LLB p < 0.05

22. Logistic Regression - LLB p < 0.05

23. Evaluation of the Model - LLB • The classification matrix for the model was examined:

24. Logistic Regression - LLB Extended p < 0.05

25. Logistic Regression - LLB Extended p < 0.05

26. Evaluation of the Model – LLB Extended • The classification matrix for the model was examined:

27. Logistic Regression - Limitations & Future Research • Consider that other biographical or non-cognitive variables could play a role in predicting “academic success” for example, motivation, study habits, self-efficacy, perseverance, and so forth. Future studies should consider gathering and adding in such variables as predictors. • Future studies should consider adding in variables from Learning Management System data as predictors. • Future studies should investigate and analyse potentially significant predictors in relation to academic success for other faculties (whether models that successfully predict “academic success” in other faculties and programmes are similar, or whether they differ from these models for these programmes in the Law Faculty).

28. Logistic Regression - Implications • Build LR “models” into the early warning system: • Once the results for the first semester test for each relevant first semester, first year Law module for the LLB and LLB Extended programmes are available, the data for the biographical and school-based predictors for each new intake of first year Law students, could then be drawn in. • The automated system could then use the logistic regression equations to calculate which students are predicted to be potentially academically unsuccessful in the first semester exams, and who are thus “in need of support.” • It would then automatically generate a list (report) identifying those students • In this way “students in need of support” in the LLB and LLB Extended programmes can be timeously identified in the first semester - while support is still possible. • These students could then be contacted by lecturers and referred to the Faculty academic advisor, who would work with the lecturers and CTLM staff, to point them towards the sorts of interventions that may best support them. • Interventions can be undertaken by Law Faculty lecturers & university support services for these identified students. • These students’ further academic progress in the semester would be monitored, and their uptake of support tracked via the system.

29. Working together - Breaking the silo’s Tells a more complete Story. Provides a broader view of students and student success. Empirical evidence to support the development & use of RADAR in the Law Faculty. Permits more comprehensive plans for support Interventions. FORMATION OF A WHOLE OIP CAAR LAW Provided a clearer description of typical law student trajectories & important predictive variables. Can support & inform CAAR practices around access to the Law Faculty programmes for students who do not meet direct entry requirements. Informed the Siyaphumelela project & development of the institutional RADAR system. Created space for exploration of new research techniques, thus contributing to professional development for all involved. Informs understanding of the student body. Can inform decisions around the development of academic support interventions to assist students in the Faculty. Points to possible areas for consideration for re-curriculation/ targeted interventions within the curriculum. KNOWLEDGE

30. Working together - Breaking the silo’s WHY? WHAT’S BEST? VALUE HOW TO? WHAT? https://kvaes.wordpress.com/2013/05/31/data-knowledge-information-wisdom/

31. Marian Neale-Shutte marian.neale-shutte@mandela.ac.za Qobo Qwaka qobo.qwaka@mandela.ac.za Andrea Watson andrea.watson@mandela.ac.za Kim Hurter kim.hurter@mandela.ac.za

32. www.mandela.ac.za