Research Designs in Education

1. Research Designs in Education Qualitative vs Quantitative Experiments vs Quasi-Experiments

2. Goals For Today Help you read more efficiently for classes like this one Help you understand the value of both quantitative and qualitative research Help you understand more about how to evaluate causal claims

3. How to Read an Academic Research Article for Classes Like This

4. Know why you are being asked to read it • Read it strategically, not front to back • For classes like mine, look for: • The main (1 or 2) questions or arguments • The basic (1 or 2) conclusions • The very basics of the research design • For other purposes, this is not enough

5. Academic research articles are almost all structured the same way • Abstract • Introduction • Review of theory and prior evidence • Description of Research Design • Presentation of Results • Discussion and Conclusions • References

6. 1 2 (maybe skim) 4 3 • For classes like mine, read in this order: • Abstract • Introduction • Review of theory and prior evidence • Description of Research Design • Presentation of Results • Discussion and Conclusions • References

8. Qualitative vs Quantitative

9. QUANTITATIVE RESEARCH Studies social phenomena through quantifiable evidence, and often relies on statistical analysis of many observations to yield valid, reliable, and generalizable claims

10. QUALITATIVE RESEARCH Emphasizes understanding of social phenomena through direct observation, communication with participants, or analysis of texts, and may stress contextual subjective accuracy over generalizability

11. QUANTITATIVE Studies social phenomena through quantifiable evidence, and often relies on statistical analysis of many observations to yield valid, reliable, and generalizable claims

12. QUANTITATIVE Emphasizes understanding of social phenomena through direct observation, communication with participants, or analysis of texts, and may stress contextual subjective accuracy over generalizability

13. Curriculum Tracking • QUANTITATIVE • Formally quantified outcome (math test scores) • Large probability sample to maximize generalizability • Description of broad and general patterns • Less attention to meaning or on-the-ground process

14. Curriculum Tracking • QUANTITATIVE • Small, strategically, smaller selected sample • Description of process and meaning, in context • Less attention to generalizable patterns than to the mechanisms that generate them

15. Racialized Peer Pressure • QUANTITATIVE • Formally quantified outcome (test scores) • Large probability sample to maximize generalizability • Description of broad and general patterns • Less attention to meaning or on-the-ground process

16. Racialized Peer Pressure • QUANTITATIVE • Small, strategically, smaller selected sample • Description of process and meaning, in context • Less attention to generalizable patterns than to the mechanisms that generate them

17. OBVIOUS POINTS Neither is “better,” more objective, more scientific, more useful, or more valid The two approaches are complementary, and help understand different aspects of social phenomena

18. Qualitative vs Quantitative LESS OBVIOUS POINTS Both kinds of research are expensive to conduct Few people are well trained to do both Arguments about which is “better” are based more on ignorance and myopia that on fact

19. Experiments vs Quasi-Experiments

20. Qualitative vs Quantitative X Y CAUSALITY How can we establish that one thing “causes” an outcome in education research?

21. Qualitative vs Quantitative X Y Z CONFOUNDING Factors associated with X also matter for Y

22. Qualitative vs Quantitative X=Parent Involvement Y = HS Completion Z=Family Income CONFOUNDING Factors associated with X also matter for Y

23. Qualitative vs Quantitative X=School Funding Y=Achievement Z=County Poverty Rate CONFOUNDING Factors associated with X also matter for Y

24. Qualitative vs Quantitative X Y Z EXPERIMENTS Value of the treatment variable is assigned only through chance

25. Qualitative vs Quantitative X Y Z EXPERIMENTS Confounding is not possible

26. Qualitative vs Quantitative X Y Z QUASI-EXPERIMENTS Value of the treatment variable is not assigned through chance … and thus risks confounding

27. Qualitative vs Quantitative X=Parent Involvement Y=H.S. Graduation Z=Family Income QUASI-EXPERIMENTS Why not always do experiments? Because you can’t always randomize the treatment

28. Qualitative vs Quantitative X=School Funding Y=Achievement Z=County Poverty Rate QUASI-EXPERIMENTS Why not always do experiments? Because you can’t always randomize the treatment

29. Qualitative vs Quantitative X Y Z QUASI-EXPERIMENTS Statistical approaches are used to eliminate bias from confounding … but there is always a risk

30. EXPERIMENT Levels of cultural capital assigned entirely by chance

31. QUASI-EXPERIMENT Levels of municipality-level homicide not assigned by chance Statistical procedures designed to minimize risk of bias from confoudning

32. NO CLASS ON THURSDAY