Dr Wai Yi Feng University of Cambridge, Faculty of Education

1. London Conference on Employer Engagementin Education and Training21-22 July 2016 Measuring Impact of Employer Engagement Programmes in STEM Dr Wai Yi Feng University of Cambridge, Faculty of Education Royal Society Ogden Education Research Fellow Mark Williams Engineering Development Trust (EDT) Education Director & Industrial Cadets Quality Assurance Manager

2. STEM Enrichment • “Activities not prescribedby the school curriculum, which aim to enhance students’ experience of STEM subjects”(e.g. Correll, 1978; Clendening & Davies, 1980) • Programmes, interventions, activities in and out of school: • Lessons incorporating “rich” tasks designed to support exploration and problem solving • School clubs • Competitions • Masterclasses • Summer schools / residential programmes • Outreach / WP activities (inc. working with scientists) • Extended investigations / projects • Mentoring programmes • Research / work placements • Designed & run by teachers in lessons / sponsored & delivered by external agencies and institutions www.etrust.org.uk www.industrialcadets.org.uk

3. EDT has engaged > 250,000 students • Track record - EDT a charity established in 1984 • Scale/reach - EDT is the leading UK-wide provider of work- related learning programmes for 11–21 year-olds and annually involves over 30,000 students • Impact - EDT inspires and motivates young people towards science, technology, engineering and maths (STEM) careers through real life industry experiences • Continuum of schemes - from half a day through to a full twelve months: • > 500 companies • > 1400 schools • > 45 universities www.etrust.org.uk www.industrialcadets.org.uk

4. Tasters Projects Placements • EDT TASTERS raise awareness of careers, inform choice and stimulate young people to consider a future in science, technology, engineering and maths • EDT PROJECTS develop skills, shift perceptions, inform career choice and raise awareness of science, technology, engineering and maths in industry • EDT PLACEMENTS develop skills, inform career choice, raise awareness and allow young people to gain workplace experience in science, technology, engineering and maths www.etrust.org.uk www.industrialcadets.org.uk

5. EDT has a range of programmes to facilitate engagement Tasters Projects Placements First Edition STEM Family Challenge 11-14 Go4SET Industrial Cadets is an industry-led accreditation providing a skills based school engagement structure led by industry and created for industry Routes into STEM 14-16 Inspire (girls) Engineering Education Scheme 16-18 Headstart 18+ Year in Industry www.etrust.org.uk www.industrialcadets.org.uk

6. Types of Impact • Support for learning in school • Enhanced understanding of STEM topics/disciplines, linked to more positive views/attitudes • Development of skills and learning processes • Personal and social development • Insights into STEM-related study/careers, leading to increased likelihood of participation Typology identified in WYF’s ESRC- & RS-funded research:Stage 1: literature review – document analysis – observation – interviews;Stage 2: development of common profile instrument – measurement of impact www.etrust.org.uk www.industrialcadets.org.uk

7. Support for Learning in School “Some of the things I have come across [in Outreach] I later met in lessons. So I understand it better when Miss introduced it in class. And because it was different – how it was taught to us [in Outreach] – I see a different side of it.” (Student on NRICH Outreach) • Helped me better understand how things I learn in lessons can be applied in the real world • Gave me a better appreciation of how the things I learn in school will be of use to me in future • Taught me new skills or techniques which I can apply in my school work • Answered questions I had which my teachers were not able to / did not have time to explain • Motivated me to persevere more with school work www.etrust.org.uk www.industrialcadets.org.uk

8. Enhanced Understanding of STEM – More Positive Views/Attitudes “I have not come across this whole areaof science – didn’t know it makes such an important difference to us – to people. […] I was interested in science before, but this has definitely made me more interested, I’d say.” (Students on Nuffield Research Placement) • Gave me insights into aspects of [STEM] I was not aware of • Gave me experience into [STEM] I would not otherwise have come across • Gave me insights into the links between different [STEM] areas • Helped me better understand how [STEM] is done • Made me like / enjoy [STEM] more • Gave me better appreciation of the contribution of [STEM] to society www.etrust.org.uk www.industrialcadets.org.uk

9. Development of Skills & Learning Processes “I have better problem-solving skills and team-working skillsnow. When I see a problem I’ve not seen before, I have a go at it and […] try to break [it down] – step by step. And I know how to manage my time and work better as a team with other people – so each team member contribute their strength.” (Student on CREST Award) • Improved my communication skills • Taught me how to learn / work more effectively • Improved my practical skills in [STEM] • Improved my project planning & management skills • Improved my report-writing skills • Improved my skills in presenting my work to others www.etrust.org.uk www.industrialcadets.org.uk

10. Personal & Social Development “I’ve made a lot of friends across year [groups]. I was quite shy when I joined, but I’ve grown as a person. I am more confident now – even giving talks to people, which I wouldn’t have thought I would do. […] The places you see are amazing. You see the world – the poverty compared with what we’ve got here.” (Student on Royal Society Partnership Grant - Conservation Project) • Made me a happier / more fulfilled person • Brought me together with like-minded people my age • Introduced me to inspirational mentors, supervisors or role models • Made me more mature as a person • Made me more motivated to do well • Made me feel a part of the [STEM] community www.etrust.org.uk www.industrialcadets.org.uk

11. Insights into STEM Study / Careers – Increased Likelihood of Participation “The mentors told us a lot about their work and how what we do relates to it and builds up – like to the context of it. I didn’t think what we do in school could relate to [work that’s done in industry]. And then, there’s so much more – like a whole new level – the stuff they have and use in the company. It made me want to have a go – maybe study it further, get to know a bit more.” (Student on Go4SET) • Gave me better understanding of expectations in work / employment • Helped me to think about what I might do after I leave school • Gave me insights into what it is like to study at university • Gave me experience of working in a university / company • Introduced me to people who gave me good careers / study advice • Made me more likely to choose to study [STEM] related subjects in future www.etrust.org.uk www.industrialcadets.org.uk

12. Questionnaire (Profiling Tool) www.etrust.org.uk www.industrialcadets.org.uk

13. Scoring (EES Student Data, n=155) www.etrust.org.uk www.industrialcadets.org.uk

14. Example Profiles www.etrust.org.uk www.industrialcadets.org.uk

15. EES: Students’ Experience vs. EDT Expectations • Similar profiles • EDT staff have good, shared understanding of EES performance, strengths, weaknesses • EES performs well relative to EDT expectations • Indicators • Exceeded expectation (>100%): • Made me like or enjoy STEM more (116%) • Increased my self-confidence or resilience (102%) • Met expectation (90%–100%): • Deepened my understanding of STEM (96%) • Gave me experience in STEM I would not otherwise have come across (91%) • Working towards expectation (75%–90%): • Taught me new skills or techniques which I can apply in my school work (87%) • Falling short of expectation (<75%): • Improved my organisational skills (73%) • Types of Impact • Met expectations (90%–100%): • Support for Learning in School (91%) • Better Understanding of STEM (93%) • Personal and Social Development (100%) • Working towards expectations (75%–90%): • Development of Skills (85%) • Insights into STEM Related Study / Careers (81%) www.etrust.org.uk www.industrialcadets.org.uk

16. Students’ Experience: EES vs. Go4SET Go4SET is intended to be a junior version of EES, targeted at a younger age group Similar profiles provide first evidence that vindicates design decisions Examination of differences enables better understanding of both schemes www.etrust.org.uk www.industrialcadets.org.uk

17. Implications for EDT • Greater and deeper understanding of theTypes of Impact • Impact assessment for existing programmes – programme modification • EDT Quality Assurance • Influence and guide new programme development • Influence and guide building of EDT programme portfolio www.etrust.org.uk www.industrialcadets.org.uk

18. Other Questions • Differences: • different models of operation • gender, background or prior experience of students • Tracking: • changes in outcome reported by a cohort over time • programme performance against programme changes over time • Benchmarking / Target setting • “Value for money” / “Best buy” ? • Create programmes that match needs using “basic building blocks” • Curate portfolio of complementary programmes within / across institutions, catering for different age groups and levels of investment (e.g. budget, time) www.etrust.org.uk www.industrialcadets.org.uk

19. Thank you For further informationon EDT: Email info@etrust.org.uk Website www.etrust.org.uk To contact presenters: Dr Wai Yi Feng -University of Cambridge wyf21@cam.ac.uk Mark Williams - EDT m.williams@etrust.org.uk For further informationon Industrial Cadets: Email industrialcadets@etrust.org.uk Website www.industrialcadets.org.uk