Quantitative approaches to analysis

1. Quantitative approaches to analysis Dr Dave Putwain Social and Psychological Sciences Dr Laura McMeeking CLIS

2. Quantitative analysis: an example • Social- Psychological factors which influence learning and achievement: • Motivation • Emotions (esp. anxiety) • Competence beliefs • Classroom environment • Prior research suggests that students can interpret achievement-based messages in different ways

3. My research questions… • Students in KS4 have reported that they can find fear appeals very anxiety-provoking: • What effect are teachers’ fear appeals having on students motivation? • What effect are teachers’ fear appeals having on students fear of failure? • Are they actually having any effect on GCSE performance?

4. My theoretical model… Fear of Failure GCSE Attainment Perceived fear appals Motivation

5. My analytic frame… • Structural equation modelling (SEM): is a test of whether the theoretical model provides a good fit to the empirical data • Multiple fit indices are used to assess model fit: • χ2 / df • RMSEA • CFI / TLI/ NFI/ IFI

6. The analysis… χ2 (310) = 569.15, p<.001, χ2/df = 1.84, CFI = .91, RMSEA = .05

7. My conclusion… • Fear appeals have competing effects, some positive and some negative: • A positive effect on GCSE maths performance by motivating students • A negative effect by increase by increasing fear of failure • When perceived by students as threatening only has a negative outcome

8. Quantitative analysis: Another example • The effects of a professional development (PD) program on student achievement • Content based maths/science PD for middle level (US grades 5 to 8) teachers • Given over 5 years with no particular pattern of enrollment in courses • Aimed at increasing teacher content knowledge

9. The theoretical model…

10. The research questions… • Is the PD programme effective in raising student achievement? • Are there differential effects for different groups of students? • Does more participation on the part of the teacher lead to greater effects on raising student achievement?

11. The analysis… • Chose the cohort control design for analysing two different groups • Students who had teachers before they began participation in the PD (comparison group) • Students who had teachers after they completed the PD (treatment group) • Went through several iterations of the design before settling on the final design (after consulting a statistician!)

12. The results… • The effects are mixed, and can only be limited to the sample of teachers/students in our study • Students of teachers who took two or more PD courses were anywhere from a 37% to 178% more likely to raise their proficiency level on the state test than students of teachers with no courses • However, teacher sample size was very small

13. What this brings us to… • What couldn’t we measure with the analysis method we used? • Is a state test the best way of measuring student achievement? • Could we have gotten a more well-rounded dataset by interviewing students about their experiences with participant teachers?

14. Conclusions… • Content based PD can be effective in raising student achievement but is very difficult to measure • Using quantitative and qualitative analyses in tandem can give richer, more interesting results • Choosing research questions first, then choosing the analysis design that best suits the question is the best way to do research