Writing with Quantitative and Mixed Methods Data

1. James Nazroo Cathy Marsh Centre for Census and Survey Research and Sociology, School of Social Sciences james.nazroo@manchester.ac.uk Writing with Quantitative andMixed Methods Data

2. Overview Starting points: Who is the audience? Telling a story Working with/in a structure and style Showing data Volume Good table manners Visualising data Writing a discussion: summary of, or theorising with, data Multivariate analysis, steps in a story or introducing complexity Mixing data and mixing writing?

3. Examiner: comprehensive and detailed Peer-reviewed journal (fellow academics): robust and innovative Policy and practitioners: clear and evidenced Conference: performance, visual Media: novel, controversial and easy Read examples before you start Who is the audience?And what kind of story do they want?

4. What are the key messages? How do these dictate the literature reviewed (the foundations for the story)? Which data are needed to tell the story, and in what order do they need to be presented (a linear narrative?)? Concluding the story Summary? Interesting and simple? or Interesting and complex? Example: explaining gender differences in depression Telling a story: the data-theory dialogue

5. Nazroo (1997) Gender inequalities in depression • Introduction, literature review • Consistency of findings on gender differences in depression • Challenge some explanations: artefact, alternative disorders and biological difference • Value of a focus on gender roles: role strain, cost of caring and identity relevant stressors • Key hypotheses • Methods: why a study of couples, why a focus on life events, sample, measures • Results • Women more likely to have an onset of depression • Difference entirely a consequence of events in the ‘domestic’ arena • Clear gender differences in domestic roles • Men under-report the significance of events in the ‘domestic’ arena, and women are more likely to blame themselves for the occurrence of such events • For couples with minimal differences in roles there are no differences in the impact of events in the ‘domestic’ arena • Discussion • Summarise findings • Theoretical implications: caring work and role identity, not a generic greater vulnerability • Consistency with other literature • Unique contribution of this paper, ability to precisely test relevant explanations • Possible limitations of this study

6. Science, positivism and social science: what does your audience expect? A classic structure, the IMRaD style: Introduction (with questions/hypotheses) Methods Sample Measures Analytical approach Results Discussion and conclusion Example, guidance from the British Medical Journal Writing within sections, and writing certain types of (precise) sentences Writing QuantitativelyWorking with/in a structure and style

7. How much data can there be? Example: tables in the ELSA wave 1 report Presenting data parsimoniously Which bits of data are needed for the story? Which bits of data are redundant in tables and figures? Volume of data

8. For clarity we prefer visual displays, and we leave out extraneous detail to focus attention on the story line To allow others to inspect and possibly reinterpret the result we want to leave as much of the original data as possible in numerical form Data must be explained, do not assume that the reader understands the methods used and how the presented data relate to this Showing data (Marsh 1988)

9. The title should be the first thing the reader looks at: Summarise contents When, where and who (date, geographical unit, and unit of analysis) Source of the data and unit of measurement Labels for rows and columns (do not use variable mnemonics) Missing data (cases, responses, items from a scale) It should always be possible to convert a percentage table back to raw cell frequencies – need base numbers (and maybe weighted bases) Show which way proportions/percents run Layout, order columns/rows to make comparison easier Which do you want the reader to compare? Which are theoretically next to each other? or Order according to size of effect Showing data: good table manners (Marsh 1988)

10. Household incomes of mixed and non-mixed children Millennium Cohort Study, sweep 1 (cohort members aged on average 9 months)

11. Household incomes for children in different ethnic groups: the implications of mixed ethnicity Millennium Cohort Study, sweep 1 (cohort members aged on average 9 months)

12. Why show graphs when they take more space for less information? Using the correct type of graph (scatterplot, pie chart, bar chart, stacked bar, line, etc) It is good practice to begin numerical axis at zero, otherwise you should clearly label the axis Use simple subdivisions of the scale, single units, twos, or (multiples of) fives Importance of layout, make comparisons easy, as for tables Examples Showing data: visualising data

13. Ethnic differences in equivalised household income 1999 Health Survey for England

14. The ethnic make-up of the UK Muslim population2001 Census

15. Visualising data: the importance of layout Fair Society, Healthy Lives: The Marmot Review (2010)

16. Visualising data: the importance of layout ‘The Figure shows the social gradient in the subsequent mortality of those that experienced unemployment in the early 1980s. For each occupational class, the unemployed have higher mortality than the employed’ Fair Society, Healthy Lives: The Marmot Review (2010)

17. Visualising data: the importance of layout ‘The Figure shows the social gradient in the subsequent mortality of those that experienced unemployment in the early 1980s. For each occupational class, the unemployed have higher mortality than the employed’ Fair Society, Healthy Lives: The Marmot Review (2010)

18. Discussion: summarising, or theorising with, data • Clear links between methods, data and conclusions • How important is it to theorise, rather than summarise? Pushing the data beyond description to explanation • Use the introduction to set the discussion up • Make connections with existing empirical literature to create the space to explore similarities and differences in findings, which then require explanation and allows for theoretical development • Example: How publically provided healthcare (the NHS) minimises ethnic inequalities

19. Nazroo et al. (2009) Ethnic inequalities in access to and outcomes of healthcare in the UK • Summary of findings: few ethnic differences in access to and outcomes of primary health care in the UK • Similarity with findings in other studies, and why there are differences with others • Contrast between these UK findings and findings in the US • Limitations: conditions covered, sample representativeness • Strengths: uniqueness of this work in methods and coverage • Concluding comments • Few ethnic differences in the UK • But marked differences in the US • Role of insurance in the US • Likelihood that differences between the UK and US are healthcare system driven

20. Summary and a question for discussion • Story-telling • Writing within a structure / working on a structure • Selecting data necessary to evidence the narrative • Displaying data • Writing a discussion • Do data speak for themselves? Discuss the contrasting experiences of data exploration and hypothesis testing; and what this means for writing – identifying stories, or building stories?

21. Multivariate modelling is inevitably complex Typically examining the influence of a variable when other variables are held constant But in real life other variables are not constant (an issue of variable rather than case analysis?) And many variables to consider, which are useful for theoretical development (part of the story) and which just need ‘controlling’ for? Translating coefficients into meaningful values, tell your reader what a ß coefficient or an odds ratio means. The significance of standard errors or confidence intervals. Build your model in a way that follows your intended narrative Example: The impact of employment transitions on mental wellbeing for older people. (Route out of work is not randomly distributed, so need for multivariate analysis.) Multivariate Analysis and Storytelling

22. Retirement and depressionA transition model for those ≤ state pension age How to write (tell) the story? • Comparison is with those who remain working • ß coefficient is the change in points on the depression scale that the transition is associated with, compared with those who remain working • Theoretically uninformative factors (age and gender) are included, but not shown • Those who remain unemployed and who retire sick are more likely to have a rise in depression score, compared with ... • Those who stop working to look after the home are less likely to have a rise in depression score, compared with ... • Those who retire have the same level of change as ... CESD score at Wave 3 of ELSA Model adjusted for gender, age and depression score at wave 1

23. Retirement and depressionA transition model for those ≤ state pension age CESD score at Wave 3 of ELSA • Same story as before, but additional rows suggest that retirement may be complex

24. Retirement and depressionA transition model for those ≤ state pension age CESD score at Wave 3 of ELSA • Differentiating retirement reveals contradictory effects, or the importance of context

25. Multivariate analysis, but can only show one variable/tell one story Scales, meaningful visual display Showing statistical significance, and/or standard errors, and/or 95% confidence intervals Example: Explaining the relationship between age and depression in later life Multivariate Analysis and Visualising Data

26. Age and depression : multivariate analysis (CES-D score > 4)

27. Age and depression : multivariate analysis (CES-D score > 4)

28. Age and depression : multivariate analysis (CES-D score > 4)

29. Age and depression : multivariate analysis (CES-D score > 4)

30. Age and depression : multivariate analysis (CES-D score > 4)

31. Age and depression: odds ratio for CES-D > 4 Darker shading p < 0.05 Paler shading p < 0.1

32. Why do mixed methods research? Pragmatics of funding and positioning, or something with an intrinsic value? The threat to our position as methodologists: from expert to novice (also for multidisciplinary research) Politics, conflicting epistemological (and ontological) positions, and contrasting explanatory logics: quantifying the qualitative or qualitatively driven? Teamworking and power: Where in the team do particular skills and experience lie? Methodological disrespect Authorship Need for seniority/mentorship in all ‘arms’ of the research All of this requires a clear and negotiated starting point The benefits and risks of doing mixed methods research

33. One question or complementary questions Triangulating or revealing different dimensions How to deal with contrasting data, theorising and addressing complexity Analysis and writing: How to handle so much (relevant) data? Write separately (an illustrative table or quote), with attempt to integrate in a conclusion/or challenge substantive or epistemological orthodoxies Do unique insights only emerge when we demonstrably integrate data and findings / or are the insights produced in more tacit ways (implicitly drawing on alternative orientations and data) / or by creatively drawing out ‘tensions’ in alternative uses of and orientations to the ‘mixed’ data? Example: is marital violence a problem of male violence against women, or of violence in relationships and violent spouses? Writing with mixed methods dataIntegrated and/or contrasting stories

34. Nazroo (1995) Gender and Marital Violence • Introduction, literature review • Feminist literature on marital violence • Survey evidence suggesting women are as (or more) violent as men • Positivist critique of feminist methodology, then critique of positivistic survey methods • Different theoretical orientations to gender-relations, patriarchy and power • Methods: combining within one study sample qualitative approaches, positivistic survey methods, and quantitative coding of meaning and context • Results • Survey methods: women more aggressive than men • Quantitative coding of meaning: men's aggression is much more dangerous, intimidating and harmful than women’s • Qualitative findings: detailed, credible and evocative accounts of abusive relationships that resonate with feminist analysis • Discussion • Summarise findings and explain contradictions • Implications for methodology • Implications for theoretical orientations • Conclusion • Highly misleading results from quantitative surveys • Critique of positivistic orientation to data

35. Decisions made very early in the research process will influence the kinds of questions you can tackle/stories you can tell. Plan early. Particularly important when working across methods/disciplines/teams. So think how evidence will be ‘integrated’ early on. How the narrative will draw on different elements/types of evidence. Is the study qualitatively or quantitatively lead, if either? How do the different forms of data / different questions complement each other? What does this mean for the kind of narrative that can be told? One person to do the final write up? And do not assume readers have strong knowledge of methods used, or share your epistemological position, particularly when using mixed methods. Some Concluding Thoughts

36. Mixed methods, discuss experiences of writing up apparently contrasting data Mixed methods, experiences of writing in teams with contrasting orientations to data More questions for discussion