Willeke Rietdijk Caro Garrett Marcus Grace

1. Evaluating the national“Action Research for Physics” programme- Follow up of impact on post-16 physics uptake Willeke Rietdijk Caro Garrett Marcus Grace Science Education Research Conference, York 3 July 2012

2. The Action Research for PhysicsProgramme (ARPP) • Research commissioned by the National Network of Science Learning Centres and the Department for Education (DfE). • Organised and managed by the nine Regional Science Learning Centres between September 2009 and February 2011. The ARP programme followed on from the findings and recommendations of the Girls in the Physics Classroom (Hollins et al., 2006) and Girls into Physics (Daly et al., 2009) projects.

3. The course • A model of professional development incorporating action research, with, as its aims: • to try out new approaches to teaching physics... • ...which lead to an increase in young people’s engagement with the subject... • ...and pursuit of physics post-16. • 3 professional development days • Main interventions in • Careers • Teaching and Learning

4. Main aims of the evaluation: i) to examine the effectiveness of the programme in changing students’ attitudes to physics, and their aspirations for further studies ii) to document and categorise the nature of effective practice across the action research case studies.

5. Reflections at the end of main study (2011) • Significant increase in participating students’ stated likelihood of taking physics post-16, but monitoring of this proved problematic (and too early at that stage) • Impact of the CPD programme on teachers’ practice came through very strongly – more confidence, more enthusiasm, improved teaching techniques • Teachers felt students were more interested, motivated and engaged; contradictory results from student questionnaires; focus group findings were mixed

6. Follow-up of the ARPP evaluation Part 1: development of 20 case studies. Part 2: research questions: • What were students’ reasons for choosing physics post-16? • What aspects of the classes and the teaching had contributed to this decision? (Focus groups) • What was the 2011 physics AS uptake compared to 2009 and 2010? • How did students feel about Physics AS so far? (particularly related to expectations and GCSE teaching experiences)

7. Research Methods Part 1, case studies booklet: Interviews with the teachers of the top 20 action research projects, to explore the details and successes of their intervention – part of the report and will return to briefly later. Part 2: Quantitative: • Student questionnaire for AS Physics students (whole of AS cohort including previous Year 11 ARPP participants) • N=147, from 6 colleges Qualitative: • Revisiting a sample of Year 11 ARPP participants now in Year 12 • 4 student focus group meetings across 3 SLC regions; 1 phone interview • All teachers participating in ARPP with a Year 11 group were asked about their physics post-16 uptake numbers in 2011 as compared to 2009 and 2010.

8. ARPP teacher interventions used in the 6 sample schools • Introducing electrical concepts before the maths. • Questioning strategies: • no-hands up and mini whiteboard strategies; • using Bloom’s taxonomy, letting students come up with and answer each other’s questions. • Increasing the relevance of physics by bringing in anecdotes about real-life physics. • Increasing the amount of small group discussions in lessons. • A variety of techniques to increase engagement, in and outside of lessons.

9. FINDINGSStudent questionnaire sample (N=147) • All year 12, AS physics students • 35% girls and 64% boys • 31% had been part of an ARPP intervention during Year 10/11 • 80% Triple science students of which 80% A or A* grade • 20% Additional science students: 65 % A or A* grade • November 2011: 11% ‘still deciding’ about continuing AS Physics choice in Year 12! • Not all students remembered correctly whether they had been part of an intervention • Focus groups: None of the students in the focus groups could remember the teacher’s specific intervention

10. 1. Reasons for choosing Physics AS *=p<.05; **=p<.01; ***=p<.001

11. Reasons for choosing physics • “Most important” reasons for choosing physics were: an inspirational teacher, interest/enjoyment of the subject, and needing it for future studies/careers • GCSE topics that motivated students to pursue physics at AS level: astrophysics and nuclear physics • Girls more often indicated they had chosen physics because they were interested in partsof the subject (as opposed to the whole subject), and because they thought the subject provides a good challenge (but differences here not significant)

12. 2. Focus group findings • Reasons for choosing physics the same as in questionnaire responses • Difference teacher GCSE/AS mentioned quite often – not as enthused by current teacher compared to ARPP intervention teacher, less engaged, more difficult explanations, more book work • Teachers pivotal in decision according to quite a few students: passion and enthusiasm rubbing off; instilling confidence, making subject more enjoyable • Simple explanations, varied ways of teaching/presenting materials, looking for ways to help digest big chunks of information were mentioned as important teaching strategies to help students understand physics and engage them.

13. 3. School data on post-16 uptake after ARPP

14. 4. Feelings about AS Physics so far (open question) • “A good challenge” (11 respondents) • Being able to go into GCSE topics in more detail and further complexity (11) • Topics enjoyed: Electricity/circuits (16), particle physics (12), practicals (11), quantum physics (7), the mathematics involved (5), SUVAT (4), and mechanics (4) “Very interesting - news ideas coming out, understand more about how things work.” “Working in smaller classes where everyone wants to participate, unlike GCSE” “You get to do the in-depth side of GCSE. You learn more complex things than at GCSE.” • Less interesting about AS physics so far: circuits and electricity (31), calculations and equations (14); materials (5); motion (5); fluids (5)  “[I enjoy] Not a lot - it is so difficult! The concepts of motion are interesting, just too hard!”  “[I enjoy] little at the moment. But it will get better, I have read the AS specifications/year outline.”  “Old content.’” “Repetitive, feels unimportant” “Same as GCSE just more work.” • Focus groups: corresponding findings.

15. Impact of ARPP teacher intervention • Statistical comparisons made: • Answers from students who were part of ARPP compared to those not part of ARPP • When they chose physics • Reasons for choosing physics (Topics? Teacher? Intervention-related (indirect)? • What contributed most to their decision to choose physics AS • What they liked about physics GCSE (trying to gauge if intervention-related) • No statistical differences in any questionnaire item between ARPP and non-ARPP students – so statistically no clear impact of the programme on reasons for choosing physics post-16 • No specific impact of the type of interventions could be found in terms of statistical differences in motivations to choose physics, or answers to the open questions. • HOWEVER: students did not seem to remember very well whether they had been part of an intervention, and did not seem to remember at all what this intervention was if they did

16. Conclusions • Some schools report a distinct increase in Physics AS numbers in the year after the ARP programme • Difficult to find impact of ARPP intervention on students’ reasons for choosing physics from students’ responses; BUT • general feedback from focus groups shows that teachers are very important in students’ choices for AS physics (passion, enthusiasm, engaging teaching techniques, instilling confidence)

17. Indirect impact? ARPP teacher participation  greater teacher confidence & enthusiasm about physics / better teaching strategies  students more enthused/motivated/confident?  more students opt for Physics AS?

18. Limitations • Small sample (147 students from only 6 schools/ intervention studies) • Findings difficult to generalise • Not easy to find statistical impact from a small sample • Self-selection of teachers

19. Beyond the programme and evaluation • Bearing in mind the effect of teacher effects on student decisions, as in the conclusions: • Impact of the programme on teachers’ CPD • Impact of the teachers’ CPD on their colleagues • Impact on the teachers’ practice and sustainability • Potential for effective teacher development via collaboration between Teaching Schools and Science Learning Centres……… how long is a piece of string?

20. Beyond the programme and evaluation • Bearing in mind the effect of teachers effects on student decisions, as in the conclusions: • Impact of the programme on teachers’ CPD • Impact of the teachers’ CPD on their colleagues • Impact on the teachers’ practice and sustainability

21. Beyond the programme and evaluation • Bearing in mind the effect of teachers effects on student decisions, as in the conclusions: • Impact of the programme on teachers’ CPD • Impact of the teachers’ CPD on their colleagues • Impact on the teachers’ practice and sustainability

22. Finally… • A case studies booklet is currently being compiled for publication – 20 most successful action research studies • Final (main) ARPP report is available from the NSLC website: https://www.sciencelearningcentres.org.uk/research-and-impact/research-reports