“Improving the Success of Students at Risk of Dropping Out: Changing Practices and Policies”

1. “Improving the Success of Students at Risk of Dropping Out: Changing Practices and Policies” Dr. Andrew ParkinAssociate Executive DirectorCanada Millennium Scholarship FoundationCanada 18th Annual 2009 European Access Network Annual Conference Changing the Culture of Campus Towards an Inclusive Higher Education June 22-24, 2009 York, UK

2. Outline • Introduction to the issue of persistence • Who leaves PSE and why? • Strategies to improve persistence • Foundations for Success • LE,NONET • Reflections on experimental research

3. Introduction to the Issue of Persistence

4. Introduction to the Issue of Persistence • Persistence: • the ability of students to continue their post-secondary studies from one year to the next and ultimately complete their programs • Access  Success • Meaningful access requires that students brought into to PSE must be successful in their studies • Low levels of persistence pose a problem... • For students: left without the credential they need • For institutions: poor use of resources and poor performance • For societies: • lower educational attainment • exacerbates social cleavages

5. An Increasingly Important Concern As a recent OECD report puts it: • “the growing portion of disadvantaged students enrolled in tertiary education makes the ongoing issue of their retention and programme completion an increasingly important concern in tertiary education.”* * Santiago, Paulo, Karine Tremblay, Ester Basri and Elena Arnal. 2008. Tertiary Education for the Knowledge Society, Volume 2. Paris: OECD.

6. Persistence: How Much Do We Know? • “We know very little about how many students drop out of programs, or why” (Rae, 2005) • Little research on the issue of persistence in PSE in Canada until recently • Previous data on persistence rates limited to institution-specific studies • Little program evaluation or evidence of what works • New research now becoming available: • Canada Millennium Foundation pilot projects and program evaluations

7. The Need for Program Evaluation “Presently…there is little evidence about the effects of institutional support programmes on student outcomes.” Paulo Santiago, Karine Tremblay, Ester Basri and Elena Arnal. 2008. Tertiary Education for the Knowledge Society, Volume 2. Paris: OECD. “It is ironic that institutions that spend so much time and money insisting on evidence-based decisions, spend so little time on research that evaluates higher education itself. Research on the postsecondary sector is inadequate and poorly publicized. This should change.” Bob Rae, Ontario: A Leader in Learning. Report and Recommendations. Toronto: Government of Ontario, Ministry of Training, Colleges and Universities (2005)

8. The Challenges of Program Evaluation • Student support programs are notoriously difficult to evaluate because: • Programs are designed and implemented without the requirements for a viable evaluation in mind • Data that would facilitate evaluation are not available • It is difficult to isolate the effects of the program from the possible effects of external factors • There is no counterfactual (no reasonable “control” group against which to compare the outcomes of program recipients)

9. Who Leaves PSE and Why?

10. Persistence Rates in Canada Overall Persistence Rates in Post-Secondary Education in Canada Source: Finnie and Qiu, Table 6b.

11. Barriers to Access and Persistence Malatest (2007), Class of 2003

12. Who Leaves PSE and Why? Self-Reported Reasons for Discontinuing Post-Secondary Studies

13. Who Leaves PSE and Why? Factors correlated with low persistence: • Poor academic performance (both secondary and PSE levels) • Low engagement • Inadequate financial aid package or high levels of debt • Uncertainty about career goals • Lower levels of parental education (in some studies) • Aboriginal ancestry • Gender (men are more likely to drop out than women) • Age & family status (older students and students with dependent children are more likely to drop out)

14. Percent of PSE Students Aged 24-26 Who Had Discontinued Their Original Stream of PSE *, By Grade Average in High School * Note: while some of these students will have discontinued their studies, others will have switched streams. Source: Shaienks, Danielle, Tomasz Gluszynski and Justin Bayard. 2008. Postsecondary Education: Participation and Dropping Out: Differences Across University, College and Other Types of Postsecondary Institutions. Ottawa: Statistics Canada.

15. Percent of Post-Secondary Students who Have Discontinued Their Studies (By Age Group) Source: YITS (Cohort B) – special calculation. * Excludes First Nations youth living on reserve

16. Who Leaves PSE and Why? • Those who persist are more likely to attempt more than one program than are those who drop out • A key difference between those who persist and those who leave is the ability to make adjustments along the way • Importance of “resilience” (the capacity of overcome obstacles, adapt to change, or to survive and thrive despite adversity) • Those who persist are able to make adjustments that help them to stay enrolled • Factors contributing to resilience in youth include supportive relationships with adults and parental expectations

17. Strategies to Improve Persistence Foundations for Success LE,NONET

18. Research on Policy Responses • Persistence should be addressed through a comprehensive approach • Need to address interconnected barriers to success • Need to work at institutional level rather than isolated policies implemented by various departments • There are many programs designed to improve persistence, but few of these are evaluated for impact • Student support programs can be implemented in the context of a research endeavour that allows the impact of the program to be assessed effectively • Confirms that programs deliver the intended benefits to students • Confirms that resources are well-spent

19. Overcoming Barriers to Access and Success (Millennium Pilot Projects) Financial incentives Community Support Academic preparation Career development activities Improving access to PSE Academic support Mentoring & cultural support Career development Off-campus community support Financial support GRADUATION

20. Tackling the Drop-Out Rates in Community Colleges through Financial Incentives and Case Management

21. Foundations for Success Research questions: • Do case manager-mediated support services increase the probability of completing a college program? • Do financial incentives in combination with case manager-mediated support services increase the probability of completing a college program?

22. Project Participation: At-Risk Factors 1) Remedial English/Communications: Measure: English Placement assessment results (slight variation within participating colleges) Intervention: students are either streamed to regular Communications or developmental course and are encouraged to undertake tutoring and related academic support 2) Mentoring: Measure: Fast-Track survey question on whether the student self-identified as someone who would benefit from a mentor. Intervention: Assignment of a mentor. 3) Career indecision: Measure: Four Fast-Track career clarity questions, with a cumulative score greater than 10 (i.e. neutral response to strongly agree/disagree). Intervention: two “Career Gear” workshops; Myers-Briggs; Strong-Campbell; one group debriefing; one one-on-one with counsellor (over 2 semesters)

23. Innovative Features of the Foundations for Success Model Post-admissions testing where responses result in actual redirection to existing services Case management approach to advise identified at-risk students (one-on-one advisement) “Case managers” follow students’ progress for two-years. This involves encouragement, identification of students’ needs and challenges, and redirection to appropriate services.

24. Random Assignment New students in 2-year programs are invited to complete the Fast Track survey, language assessment and sign informed consent, making them eligible for the study Students deemed at risk on at least one item Students not deemed at risk (no further involvement in project) Random Assignment occurs Program Group 2 SERVICES PLUS Academic, mentorship and career exploration support and financial incentives Program Group 1 SERVICES Academic, mentorship and career exploration support Program Group 3 COMPARISON

25. Number of Participants per College

26. Year 1 ResultsParticipation in FFS Activities Service group significantly more likely to participate in any FFS activities compared to Control group Service Plus group significantly more likely than Service group to participate in any FFS activities Source: 2008 College Administration Data, n=2,008.

27. Participation – Sub-group Analysis • Higher Semester 1 participation rates among ESL students, first generation, older, low-income and low-confidence students. • Higher Semester 2 participation rates among ESL students, older students, women, low income and low-confidence students.

28. Semester 1 GPA Service Plus group GPA was higher (2.13) compared to the Control group (2.01) Source: 2008 College Administration Data, n=2,008, missing GPAs were imputed using a regression model.

29. Semester 2 GPA Service Plus group had a higher GPA in Semester 2 (1.96) compared to the Control group (1.79). Source: 2008 College Administration Data, n=2,008, missing GPAs were imputed using a regression model.

30. Student Retention – Beginning of Year 2 Service Plus group significantly more likely to still be in program (67.2%) compared to Control group (62.6%). Adjusting for the students that did not participate in any FFS activities, the adjusted effect of the Service Plus treatment is a 6.4% increase in retention one year into students’ college program. Source: 2008 College Administration Data, n=1,711.

31. Sub-Group Analysis – Who Gets the Most Benefit from the Interventions? Retention:Service Plus ESL students, low-income students, and students with high school grades of 65 or lower show significantly higher retention rates than similar students in the Control group. Source: 2008 College Administration Data, n=1,711, Cohort 1 only.

32. Improving the persistence rate of Aboriginal learners at the University of Victoria “Success after enduring many hardships” (Salish)

33. LE,NONET • Pilot project to test the effectiveness of initiatives to improve the retention of Aboriginal students at the University of Victoria • Project recognizes that improved outcomes necessitates change not only in students but also in the university • “the onus for adjustment” should not be placed solely on Aboriginal students • LE,NONET project involves the creation and implementation of a series of programs and support structures: • Student Mentoring • Bursaries (regular and emergency) • Community Internship • Research Apprenticeship • Staff and Faculty Aboriginal Cultural Training (SFACT)

34. LE,NONET Research question: • will a series of interventions involving financial, academic, peer and cultural support have a demonstrable effect on Aboriginal students’ likelihood of completing their program of studies? • is any particular type of intervention more effective and do the interventions have greater impact when delivered in combination with one another?

35. The Research Framework • Research will document the process through which changes are proposed, developed, implemented, and modified at the university, faculty, department and program level over the course of the project • Since one of the objectives of the program is to change the university culture, it is possible for the program to affect non-participants as well as participants (cannot use random assignment) • evaluate the direct effects of participation in the various programs offered by the LE,NONET project • evaluate the indirect effects that such programs might have on students who decline to participate • Comparison of the persistence of Aboriginal students during the periods before and after the program was introduced • The LE,NONET approach will allow students to determine the depth of their involvement in the programs — making differing ‘levels of exposure’ to the interventions a key variable of interest.

36. Participation • Number of participants (2006 & 2007): 139 • Bursaries = 87 • Mentor = 17 • Mentored = 35 • Preparation seminar = 44 • Community intership = 24 • Research apprenticeship = 17 • Historical cohort: 997 • Non-participants: 728

37. Interim Findings The majority of participants (77%) reported that their participation in LE,NONET program components contributed to the development of their sense of self as an Aboriginal person. A strong majority (87%) agreed that their participation in the programs contributed to their sense of connection to the on-campus Aboriginal community. Just over half (54%) said that their involvement with LE,NONET also contributed to their sense of belonging at the university.

38. Interim Findings: Return to School Overall, about half of the research participants said that LE,NONET had contributed to their decision to return to school the following year. For some students, the financial support they received through the Project made it financially viable for them to afford their tuition and living expenses. Other respondents said that returning to school was made easier by the emotional support and sense of community provided by LE,NONET.

39. Registration Status (Interim): LE,NONET

40. Reflections on Experimental Research

41. Reflections on Experimental Research • These projects show that student support programs can be implemented in the context of experimental research initiatives that will lead to an increase stock of evidence about what may or may not work. • Experimental research is possible but not always easy to put in place. Obstacles include: • Money • Time • Confusion about the objectives of the research • Tensions between educators and researchers • Sticking to the script • What is the measure of success? • Recruitment of participants • All of these obstacles can be overcome, but only when researchers develop effective partnerships with educators and administrators.

42. Canada Millennium Scholarship Foundation1000 Sherbrooke West, Suite 800Montreal, QC H3A 3R2 1-877-786-3999 All our publications are available online: www.millenniumscholarships.ca