Engaging students with physics: an evaluation of the national Action Research for Physics programme

1. Engaging students with physics: an evaluation of the national �Action Research for Physics� programme

2. The Action Research for Physics (ARP) programme Research commissioned by the National Network of Science Learning Centres and Department for Children, Schools and Families. Organised and managed by the nine Regional Science Learning Centres between September 2009 and February 2011. The ARP programme followed on from the findings and recommendations of the Girls into Physics project (Daly et al., 2009).

3. The course A model of professional development incorporating action research, with, as its aims: to try out new approaches to teaching physics... ...which lead to an increase in young people�s engagement with the subject... ...and pursuit of physics beyond GCSE level.

4. Programme content 3 separate CPD sessions of one day, focusing on: action research theory and explanation; physics teaching strategies; feedback from individual teacher�s action research findings 2 rounds of action research in between the 3 sessions Intervention in one of the 6 strands from the Girls into Physics project: Careers, Teaching & Learning; School Culture; Progression; Classroom Management; Workforce Intervention to be developed over 2 rounds of action research to incorporate feedback from first round in second round

5. Main aims of the evaluation: i) to examine the effectiveness of the programme in changing pupils� attitudes to physics, and their aspirations for further studies (where possible, to follow pupils and ascertain actual numbers continuing beyond year 11 with physics studies); ii) to document and categorise the nature of effective practice across the action research case studies.

6. Examples of teacher projects Interventions in the area of Teaching and Learning covered the following main areas: Questioning techniques Collaborative work / experimenting with single-gender/mixed-gender groups; grouping according to level (single or mixed) Generally bringing in more real-life context into the lesson More practical work Working with concept maps Use of more/new/creative/different/visual �materials, including videoing and filming Bringing in more cutting edge/wow! physics and addressing �the big questions� Including more discussion Going outside the classroom more Bringing in cross-curricular activities/lesson plans

7. Examples of teacher projects Careers and Guidance interventions often consisted of the following activities: Getting outside speakers in; setting up physics clubs Letting students research careers and/or do presentations about what physicists do Posters/displays up in corridor/classrooms and regular updating of these

8. Examples of teacher projects Interventions in the area of Teaching and Learning covered the following main areas: Questioning techniques Collaborative work / experimenting with single-gender/mixed-gender groups; grouping according to level (single or mixed) Bringing in the context and applications at the beginning of a topic Generally bringing in more real-life context into the lesson More practical work Working with concept maps Using video peer assessment strategies Use of more/new/creative/different/visual �materials Bringing in more cutting edge/wow! physics and addressing �the big questions� Including more discussion Reducing textbook work Going outside the classroom more Bringing in cross-curricular activities/lesson plans Careers and Guidance interventions often consisted of the following activities: Getting outside speakers in; setting up physics clubs Letting students research careers and/or do presentations about what physicists do Posters/displays up in corridor/classrooms and regular updating of these

9. Research Methods Quantitative: 3 Pupil Questionnaires 2 Teacher questionnaires �Year above� Control group pupil questionnaire Teachers� Senior Managers Questionnaires Qualitative: 2 rounds of Pupil Focus group meetings Teacher focus group meetings at each SLC at the end of the programme to assess impact of taking part in course 1:1 Course tutor interviews about their views on the success of the programme and any recommendations for future running of the course

11. Pupil survey questions Interest Difficulty (including words/terms used) Relevance Careers Intention of progression to post-16 Gender How often links made in physics classes with: Other subjects Everyday life Worldwide issues How much time spent in classes on discussion/ thinking time Pupil Questionnaire 2+3: More/less interested/difficult than before? Most interested in recently? Any other comments on your physics lessons over the last 6 months?

12. Pupil survey questions Interest in physics Experienced level of difficulty of physics (and words/terms used) Views on relevance of physics (open) Careers talks; career plans Intention of choosing physics post-16 Physics lessons aimed more at boys/girls/both? How often links made in physics classes with: Other subjects Everyday life Worldwide issues How much time reported in classes for: Discussion Reflection (�Time to think things through properly�) Pupil Questionnaire 2+3: More/less interested/difficult than before? Most interested in recently? Any other comments ab0ut your physics lessons, particularly over the last 6 months?

13. Teacher survey questions Main concerns about teaching physics What needs changing in your classes to encourage post-16 Teaching strategies used in physics lessons 11 items; repeated in final questionnaire for statistical comparison of answers pre-post course; e.g.: Encouraging dialogue Differentiation for genders/abilities Links with careers, physics topics, other subjects, everyday lives of pupils, global/social challenges Teacher Questionnaire 2: Action research intervention Views on success and impact of course (on pupils, classes, teacher) Changes in practice, changes to SOW, remaining concerns Impact on colleagues/Department and wider school level

14. Teacher survey questions Demographics Open questions about: Main concerns about teaching physics What needs changing in your classes to encourage post-16 take-up of physics Teaching strategies used in physics lessons (�How often do you...� ) 11 items; repeated in final questionnaire for statistical comparison of answers pre-post course; i.e. Encouraging dialogue Differentiation for genders/abilities Links with careers, physics topics, other subjects, everyday lives of pupils, global/social challenges Teacher Questionnaire 2: Action research intervention Views on success and impact of course (on pupils, classes, and teacher) Open questions about changes in practice, changes to SOW, remaining concerns Impact on colleagues/Dept and wider school level

15. Senior Managers Questionnaire questions Reasons for authorising teacher participation in ARPP Nature and impact of teachers� action research intervention (finding out how much they are aware of this) 9 impact of course items (pupil, teacher, department, school) Usefulness of action research Usefulness of teachers� participation in ARPP Intentions of using teachers� findings in future staff CPD, sending other teachers on the course, recommending course to other CPD managers

16. Statistical relationships calculated Pupils: Interest in physics Experienced difficulty level of physics Physics careers talks reported Post-16 take-up of physics intention Lessons aimed at which gender Amount of links reported with Other subjects Everyday life Worldwide issues Gender differences in: Interest Difficulty level experienced Intention to take up physics post-16 Reported level of discussion time/reflection time Lessons aimed at which gender

17. Between Pupils and Teachers Matching what teachers say they do in their physics classes and what pupils report happens in physics lessons: Links made in classes with everyday life, other subjects, and worldwide issues; physics careers talks, amount of dialogue/discussion reported 2. Statistically comparing Pupil interest; Pupil experience of difficulty level of physics; Intention of post-GCSE take-up of physics; Gender of teacher Teacher strategies (i.e. Differentiation, dialogue, links with everyday life, other subjects, and worldwide issues, physics careers talks, amount of dialogue/discussion reported) Gender of teacher (Sometimes splitting the pupil file into girls and boys and then do the same comparisons)

18. For final Teacher and Pupil Questionnaires also: Statistical comparison of Teacher Action Research intervention areas (6 strands) Teacher indications of level of impact of interventions on Pupil Teacher With Pupil interest (+ increase) Pupil experience of difficulty level (+increase) Pupil intention of physics post-16 take-up Reported amount of discussion/reflection time in classes

19. Pupil Questionnaire 1 (2050 completed; 58% female; 45% Yr 10s, 42% yr 9s) Only 14.4% are �very likely� or �definitely� going on to study physics after their GCSEs, girls only even lower percentages: 10.8% Only 25% of pupils claim to have had talks about careers in physics, mostly with their science teacher and/or a parent. 30% of pupils feel that there is not enough time in physics lessons for discussion 39% report that there isn�t enough time to think things through properly (girls 45%) Pupils are significantly more interested in physics when physics is linked to their daily life, worldwide issues, or to other subjects. Pupils are most interested in physics when lessons are aimed at their own gender.

20. Teacher Questionnaire 1 (110 completed; 56% female; 55% with a physics subject background; 65% with a physics-related degree) 89% said they often encourage dialogue between teacher and pupil 59% often encourage dialogue between pupil and pupil. 90% never differentiate for boys and girls Over 80% use gender-neutral examples and illustrations, use non-technical language and analogies, make links with everyday life, other subjects and global and social issues 74% make links with careers (but only 25% of students report this!) Most frequent concern mentioned by teachers is ability to engage and enthuse pupils and making physics relevant (by 28% of them) Similarly, the most frequent aspect mentioned that needs changing in their classes by physics teachers is increasing the relevance of physics (by 38% of them).

21. Links between pupil and teacher variables The more often teachers link physics with everyday life, the more interested pupils are in physics. (Not for links made with other topics, global/social issues, careers in lessons!) A significant negative relationship between how often teachers encourage dialogue between pupils and themselves and how interested pupils are in physics, and how easy they find physics! Very strong positive correlation between how often teachers indicate that they link physics with everyday life and with careers, and how often pupils feel that this is happening - pupils are clearly aware of their teachers� efforts in this respect. Pupils find physics significantly easier when teachers indicate that they link physics with global and social issues more often Girls find physics significantly less difficult, and consider the post-16 take up of physics more, when their teacher is female � no gender relationship for boys Pupils did not find physics easier or more interesting when their teacher had a physics degree! (Biology-easier; engineering-more interesting (boys)

22. Pupil Questionnaire 2 (after teachers� 1st rounds of action research) (921 (37 classes) completed; 60% female) Almost a third find physics more interesting, and less difficult than before The percentage saying physics is �quite difficult� has dropped from 52% to 44% between the two measurement points, a significant effect. Pupils report they have had far more talks about careers in physics in their lessons (49% from 27%) - a significant change. Significantly more pupils are now saying that physics is often linked to their everyday life and worldwide issues in classes. Significantly more pupils now feel that the amount of thinking time is �about right�.

23. Pupil Questionnaire 3 (after teachers� 2nd rounds of action research) (958 pupils (38 groups) completed; 58% female) More than a third find physics more interesting, and 25% find it less difficult than before (20% find it more difficult!) Girls keep finding physics significantly less interesting and significantly more difficult than boys (words/terms used in physics as well) Girls now seem more decided about choosing physics post-16 (positively or negatively; boys show more intention of choosing it; % girls �very likely� or �definitely� choosing physics post-16 has also slightly increased) Pupils are now most interested in their physics lessons when they feel these are aimed equally at both genders and least when they are aimed at the opposite gender Pupils are now significantly more interested in physics when teachers make more links with global and social issues more (before this link was only with everyday life)

24. Comparisons between PQ1, 2 and 3^ ^Significant difference in answers to question between baseline and interim measurement *Significant difference in answers to question between baseline and final measurement **Significant difference in answers to question between interim and final measurement 

25. Final Teacher Questionnaire 64 completed this out of 101 teachers starting the Programme 56.3% were �very positive� about taking part in the ARP Programme, 42.2% �positive� 95% felt it had been effective in increasing their pupils� engagement with physics (25% �very effective�; 70% �quite effective�) 64% felt an increased motivation/enthusiasm to teach physics since starting the programme; 50% said their own interest in physics had increased 86% have made changes to their classes as a result of the programme as a whole; 50% have made changes to their schemes of work 72% increased discussion time; 58% increased reflection time 91% feel they have learnt quite a lot, or a lot about action research 100% find action research useful for improving classroom practice Many teachers (39%) say their learning is being applied by their colleagues, and also on a wider school level in 31% of cases, and in other schools in 11% of cases These teachers significantly more often than before the Programme: Link physics with everyday life in lessons Use gender-neutral examples

26. Increase in teacher confidence levels Large percentages of these 64 teachers are now feeling more confident than before the Programme in: Getting pupils/girls engaged in physics (78%/56% feel more confident) Making physics relevant (78%) Boosting pupils�/girls� confidence in physics (63%/59%) Increasing pupils� awareness of careers/futures in physics (70%) Making abstract physics more �visible� for pupils (70%) Making the most of resources (62%)

27. Links between teacher effects of ARPP and student interest [statistically sig links]

28. Comparisons between Final Pupil Questionnaire and the Control (�Year Above�) group 656 control group respondents Compared only between schools of which Final Pupil Questionnaire received Significant differences of Final Pupil Questionnaire with Control Group: Reduced experienced level of difficulty of physics, and of words/terms used Increase in pupil interest in physics Increase in likelihood of post-16 physics uptake Increase in reported reflection time Higher number of reported careers talks, esp. by science teachers

29. Limitations and issues encounteredduring the evaluation When did teachers actually give questionnaires to their pupils? In some schools, the information gained applied to physics lessons of another teacher as the ARPP teacher had not started teaching this group physics Some teachers reported pupil �questionnaire fatigue� Not all teachers actually did a 1st round of action research, or had a different topic during the 1st round, but still sent baseline and interim questionnaires Sometimes it seemed in the pupil focus groups that the teachers had already long finished their interventions and the pupils couldn�t really think what might have changed � might apply to final questionnaires as well In the case of career interventions, these were often �bolted on� outside of classes so pupils might not have found changes in their classes Spurious factors! 68 teachers carried out unique projects in their unique circumstances � the more generalised measurement of the success of the course/interventions as a whole is therefore subject to many factors and we cannot strictly refer to the evaluation of �one intervention�. There was some variability in the delivery across the SLCs

30. Summary of Focus GroupMeetings Prior to interventions �Physics deals with unanswered questions; it is not common knowledge, we don�t know everything about it yet � which makes it fascinating� �Boring� aspects often mentioned are: circuits, friction, and a feeling of lots of repetition. Maths is most often mentioned as a difficult aspect. Frustrating aspects often mentioned are: the lack of deeper explanations about how things work, and that in physics it is harder to �see� things happening (especially by girls). Physics practicals are generally very much enjoyed, and found helpful. Most groups say careers are hardly mentioned in class and that they don�t have much of an idea what you can do with physics. Girls more often feel that physics is a male-oriented profession and are not eager to enter into if there are few women there. In many groups it is felt there is quite a lot of time for discussion in class. Many feel that they should be taken out of the classroom more � more activities around the school, but also physics trips are much desired.

31. Teacher focus group meetings 9 held at end of CPD day 3 at all Science Learning Centres (live or VC) Teachers very positive about the benefits for their practice of discussing classroom strategies and ideas with other teachers on the course Many teachers feel formal engagement with action research has had many benefits realise they do it all the time, but the course ensured greater focus, and the cyclical nature of action research impacted more on their practice some say they will try to do this more often Many teachers feel pupils are more engaged now Discussion in class is found to be extremely useful Many teachers not sure of impact on post-16 take-up (hard to tell � and even if increased take-up, what are the reasons?)

32. Recommendations/implications Engagement with course and action research has been a driver of change in the classroom If the course were to be repeated/continued, lessons have been learnt about timing and organisation Tracking of participant teachers and students is desirable

33. References Daly, A., Grant, L. and Bultitude, K. (2009) Girls into Physics: Action Research: Evaluation Report, London: Department of Children, Schools and Families, UK Hollins, M., Murphy, P., Ponchaud, B. and Whitelegg, E. (2006) Girls in the Physics Classroom: A Teachers� Guide for Action. London, Institute of Physics Murphy, P. and Whitelegg, E. (2006). Institute of Physics Report: Girls in the Physics Classroom: A Review of the Research on the Participation of Girls in Physics. London: Institute of Physics All accessible from the IoP website