* Aston University **University of Ulster

1. Student Engagement and Learning: The Pedagogical and Practical Value of Formal Work PlacementsAndrews, J*, Green, J.P**, Higson, H.E*, and Jones, C.M* * Aston University**University of Ulster

2. Background • In the White paper (2011) Higher Education: Students at the Heart of the System the Government affirms that students should expect to receive excellent teaching. • One of the dimensions identified by Gibbs (2010) of a high quality learning experience is the levels of student effort and engagement. • The White Paper (2011) also highlights the decline in students undertaking a “sandwich” degree, despite the views of educationalists and employers on the benefits of a placement year (Confederation of British Industry, 2009), which is in part attributed to students feeling that “the extra year of study was not producing enough added benefit.” (White Paper, page 41). • The Wilson Review (2012) A Review of Business – University Collaboration, recommends that: “Ideally, every full‐time undergraduate student should have the opportunity to experience a structured, university‐approved undergraduate internship during their period of study.” (Wilson Review, page 40).

3. Specifically in Northern Ireland “The Department expects the institutions to ensure that all learners have the opportunity to undertake a period of work placement whilst undertaking a higher education course. This includes work experience, voluntary and community work and work outside term time, all of which may be either directly or indirectly linked to their course.” (Graduating to Success, 2012)

4. So what are the benefits to students? Placements and degree performance: Do placements lead to better marks? Do more “engaged” students complete a placement year? Do “weaker” students benefit more from placement?

5. Evidence

6. What is a better student? • From the literature a “better student” is one who performs better in either pre-university or prior university assessment. • Performance is a function of ability, effort and reward. • Is a “better student” not an “engaged student”? • Newman (1992) argues that student engagement occurs when “...students make a psychological investment in learning. They try hard to learn what school offers. They take pride not simply in earning the formal indications of success (grades), but in understanding the material and incorporating or internalizing it in their lives”.

7. What is a “better student”? • The use of pre-university and prior university performance as measures of engagement is problematic if it is accepted that the extrinsic rewards are different. • If extrinsic reward is perceived as degree classification (which evidence suggests is linked to graduate level employment, Moores and Reddy, 2011), effort on pre-university education a priori is likely to be high to gain university entry. In first year, which for most UK universities (including Aston and Ulster) does not contribute towards degree classification, a priori it is likely that the extrinsic reward is lower, but for engaged students making a “psychological investment in learning” there would be no change in effort.

8. The Key Question? • With regard to the placement year, if only engaged students complete this then it could be argued that the finding of previous studies that students who complete placements perform better in final year is really that students who are engaged throughout their course of study perform better in final year. In other words the completion of placement, “the placement effect” is a surrogate for student engagement, “the engagement effect”. • In final year which contributes the most to degree classification for the vast majority of UK degrees, it may be argued that the effort of all students and the level of engagement will be at a maximum.

9. Statistical issues • In two random samples that are drawn from identical populations, where the only differentiating factor is whether a student has completed a placement or not, then providing the sample sizes are large enough for the central limit theorem to apply, ANOVA, standard OLS regression, or indeed a simple comparison of means will provide a test of whether the effect of a placement is significant. • If the two samples are drawn from different populations, however (more engaged and less engaged students through out the entire course of study), which is the issue under investigation in this study, then neither ANOVA or OLS regression will allow for sample selection.

10. Statistical issues • This issue is then a classical example of a sample selection problem, as considered by Heckman (1979) in the context of investigating the determinants of wage offers (in this study final year degree performance) but only having access to data with regard to those employed (students in final year, irrespective if a placement year has been completed, who are engaged to achieve a high degree classification). • Heckman’s (1979) two stage procedure involves estimating a probit model to account for the probability of an individual undertaking a placement. The second stage estimates the performance model including the selection parameter (Mills ratio).

11. Methodology • To avoid bias • employed sample selection model developed by Heckman (1979). • aimed to find a characteristic correlated with likelihood of students choosing to take placements, but uncorrelated with their ability • used the background of the student, measured by the National Statistics Socio-economic Classification (NS-SEC) of the parents • Approach • employed a general to specific modelling strategy; started with variables that conceptually or theoretically may be significant. • the model was then tested systematically and insignificant variables were removed • tested for subsequent loss of predictive power, until arrival at the final specifications.

12. Sample • For Aston approximately 5,700 student observations are included in the final sample, over the graduation years 2001-2010, covering courses across the university’s five Schools: (1) Aston Business School (ABS); (2) Inter-disciplinary Studies (IDS); (3) Engineering & Applied Sciences (E&A); (4) Languages and Social Sciences (L&S); and (5) Life & Health Sciences (LHS). • For Ulster 737 student observations are included in the final sample covering graduation in 2009 and 2010 most of which derive from the Ulster Business School.

13. Descriptive statistics Aston (Table 1) • A total of 39% of students from Aston included in the analysis engaged in a work placement year. • Out of the base population of students included in the study, 10% of them from Aston are classified as mature students (above the age of 21 when their course begins), 50% of them are female, 79% of them are British and 94% of them have no disability. • In terms of parental class, 7% of students come from the Higher Managerial Class, 11% from the Intermediate class and nearly 30% from the Lower Managerial Class. • Finally, in terms of school background, 32% of them came from a comprehensive school, 6% from Grammar School and 10% came from an Independent School.

14. Descriptive statistics • A total of 66 per cent of students went on work placement. • There are almost no mature students at Ulster (the University of Ulster offers many of its degrees on a part-time basis, which tends to attract mature students) for this reason to investigate the impact of “relative maturity” a variable that identifies a student’s age at the start of their course is investigated. The average age is 19.5 years. There are more females - 68% - and almost all of the students are from Northern Ireland. • The pattern of class distribution reflects the Aston data with a significant number of students coming from the Intermediate and Lower Managerial classes. • In terms of schooling, 36% of students come from Grammar schools, reflecting Northern Ireland’s option to retain such schools in the 1970’s and 1980’s, and 54% of students studied at comprehensive school.

15. How well can the probability of a student taking a placement be predicted? (Table 2) • For Aston, mature students are 7% less likely to take a placement compared with students under the age of 21. • For continuous variables, the marginal effect shows the increase (decrease) in probability of a student, of a given type, undertaking a placement resulting from a one unit increase in the variable concerned whilst holding all other variables constant at their means. For example at Ulster, as a student becomes older by 1 year they are 15% more likely to go on placement. Interestingly these results somewhat contradict each other but it is important to note that the variable Mature in the Aston model is very different to the Age variable in the Ulster model. • Typically there were almost no ‘mature’ students included in the Ulster data. Thus the positive coefficient suggests that older students, who are not over the age of 21, and are therefore not defined as mature, are more likely to undertake a placement.

16. How well can the probability of a student taking a placement be predicted? (Table 2) • Gender plays no role in determining the choice as to whether to go on placement – the coefficients are statistically insignificant. In terms of parental class, students with parents that work in the Higher Managerial class appear to be more likely to undertake a placement at both Aston and Ulster. • The level of engagement is measured by a student’s average score in their 1st year which does not contribute to final classification at Aston or Ulster. As can be seen, for both Aston and Ulster the coefficient estimate for the 1st year average is highly significant. This suggests that “more engaged” students do choose to go on placement.

17. Do “engaged students” choose to go on placement? (Table 3) • The conclusions that can be drawn are relatively robust across both Aston and Ulster. The selection term (Mills Ratio) is insignificant suggesting that there is no self-selection present. The coefficient on the placement dummy is positive and highly statistically significant at 0.053 for Aston and 0.060 for Ulster. This suggests that undertaking a placement improves the performance in finals by 5.2% for Aston and 6% for Ulster. For many students this might amount to a difference in degree classification. One observable difference between Aston and Ulster concerns the impact of gender, which is statistically significant for Ulster. Although it should be noted that the Ulster sample has a much higher proportion of female students compared to Aston which is roughly 50:50. • The overall conclusion which can be reached is that “more engaged students” as measured by first year performance do choose to go on placement (as shown by the probit model) but regardless of the level of engagement the completion of a work placement year improves the students performance in the final year.

18. Is there heterogeneity in the Aston results? (Tables 4 & 5) • Performance in the 1st year (the measure of student engagement) has a strong and statistically significant impact on a student’s decision to do a placement. This effect is stronger in L&S and IDS compared with the other Schools. These results show quite clearly that engaged students are more likely to go on placement. • The Mills ratio is insignificant for each School. This suggests that there is no selection effect at the School level. However, the coefficient for the placement dummy variable does vary across schools. • Nevertheless, the conclusion is the same as the general results above: Engaged students do choose to do placements but this does not matter, a placement year improves final year performance.

19. Is the benefit of placement the same for all students? (Table 6) • This question is easily analysed by splitting the data in to separate samples – one sample for students who obtain an average of 60% or higher in their second year, and one sample for students who obtain an average of less than 60% in their second year. In other words we are assessing which students benefit more from the placement. • As no sample selection effect has been identified standard OLS regression is employed. The dependent variable is simply the average final year mark and the primary interest is the coefficient for the placement dummy variable. For each sub-sample of students it is clear that going on placement improves performance, the variable is positive and statistically significant for both Aston and Ulster. It can therefore be concluded that students achieving a 2:2 or less in the year prior to placement gain more from a placement compared to 2:1 or higher students.

20. Does the “quality of placement” matter? (Table 7) • Placement quality is measured as the actual mark achieved in the placement year. This means that the sample is now restricted to placement only students. • Again self-election is ignored by excluding the Mills ratio from the analysis. In addition, some students are omitted from the analysis for Aston because their placement mark is either “pass” or “fail”. • The authors accept that placement mark may not be an ideal way of measuring the quality of a placement. However, it is argued that the higher the placement score the more engaged the student has been on placement. • The results suggest that increased placement performance has a positive impact on the performance in final year. The coefficient estimates for both Aston and Ulster are positive and significant however, they are quite small in magnitude.

21. CONCLUSIONS: • The results of this study indicate that improved final year performance is observed for students undertaking a work placement year. • Further, “more engaged students” as measured by first year degree performance do complete a placement year. However there is no evidence of sample selection bias, “the student engagement effect”. • The evidence presented suggests that irrespective of the level of engagement, the completion of a placement year “per se” is associated with better final year performance. I • In addition those students performing less well in prior university assessment benefit more from taking a placement year in terms of improved final year performance and the “quality of the placement” is also related to final year performance.

22. Further work • Follow up qualitative studies have been conducted at Ulster and Aston.

23. Further information Professor Helen Higson h.e.higson@aston.ac.uk