Rose Lindsey and Liz Metcalfe, University of Southampton Third Sector Research Centre

1. Mixing science and intuition: the process of synthesising data from a longitudinal mixed methods study of volunteering Rose Lindsey and Liz Metcalfe, University of Southampton Third Sector Research Centre ESRC grant no. ES/K003550/1

2. Presentation aim • To explore the challenges encountered when combining longitudinal qualitative and quantitative secondary data to study volunteering across time • Key challenge: What do we mean when we talk about synthesising, integration, combining, mixing, interweaving, blending, merging…? (Bryman, 2008) • Do we think our methods of combining have actually worked?

3. Presentation outline Part 1: Designing the project • Introduction to Continuity and Change project • Discussion of the methodological challenges faced within the mixed-method research design • Bringing different data sources and findings into dialogue Part 2: Challenges in practice • Exploring the analytical challenges faced when putting the design into practice: • Working across methodological paradigms • Understanding the effect of, and working across, time • Design versus practice

4. Part 1: The Continuity and Change Project • Aim: To explore individual attitudes and behaviours towards volunteering, and individual views on the role and responsibility of the state towards provision for social need, across a period of thirty years. • Design: Concurrent use of longitudinal mixed-methods to analyse secondary data • Time-frame: 1981-2012, encompassing different periods of economic adversity and prosperity • Project website: http://longitudinalvolunteering.wordpress.com

5. Choice of secondary data sets Qualitative data Quantitative data • The Mass Observation Project • Aim: to capture experiences, thoughts and opinions of individuals • A national panel of volunteers writing in response to themed questions or ‘directives’ (1981 to present day) • Longitudinal data following the same people through thirty years of their life-course • British Household Panel Survey/Understanding Society • Aim: to understand individuals’ and households’ social and economic change • A national panel of the British population and volunteers (1991 to 2012) • Longitudinal data • British Social Attitudes Survey • Aim: to track people‘s changing social, political and moral attitudes • A national survey of the British population (1983 to 2012) • Cross-sectional data

6. Why did we use mixed-methods?Enhancing strengths and offsetting weaknesses Our research design aimed to potentially ‘offset’ the respective weaknesses of these two analytical methodologies by taking advantage of their joint strengths to provide a ‘complete[ness]’, and ‘comprehensive’ picture (Bryman, 2008, p.91)

7. Multi-layered picture of volunteering behaviour In-depth analysis of individual volunteers Contextual: social, economic and political events over time Behaviour and attitude analysis for volunteers within the population Focus on individuals increases Sample size decreases • In-depth analysis of individual volunteers Contextual: social, economic and political events over time

8. Bringing secondary data sources, analyses and findings into dialogue We aimed for three types of mixed-method dialogue: • across the lifetime of the project, described by Tashakkori and Teddlie (2008, p.104) as a ‘continuous feedback loop’, to enable an iterative research process; • some direct comparisons between qualitative and quantitative analyses where there was a fit between the data; • combining substantive findings so that the sum of our joint knowledge claims would be greater than our individual findings Qualitative data Substantive findings Quantitative data Project beginning Project end

9. Study design challenges-sample fit The Mass Observation Project British Household Panel Survey/Understanding Society • The Mass Observation Project (MOP) • 15 directives (sets of questions) were selected • N=38 • 2 samples were taken to provide a range of ages and occupations • Sample 1, n=20 were writers from 1981 to 2012 • Sample 2, n=18 were younger and wrote for shorter periods of time • Sample restricted to available volunteering data (every other year between 1996 to 2011) • N=2067 British Social Attitudes Survey • Questions of volunteering only asked a limited number of times • Number of people each year mean (sd) 3393 (711.7)

10. How the three datasets complement each other, temporally and thematically

11. Part 2: Challenges in practice Three main challenges were present throughout the project: • Working across methodological paradigms • Understanding the effect of, and working across time • Putting the design into practice

12. Working and communicating across methodological paradigms Working across methodologies we encountered some challenges: • Differences in terminology • Forming definitions • Timings/speed of analysis • Methodological standpoints: differences in the types of questions that are being addressed • Conceptions of time

13. How time fits together The way that these multiple perceptions of time interact and intersect (or not) was at the heart of the mixed methods effort for our research project. • the flow of personal biographical, narrative, retrospective, life-course, individual time • chronological time, moving from one year to the next • contextual public/collective time related to chronological time Hi, I’m Sarah Children Retirement Marriage 1981 1984 1987 1990 1993 1996 1999 2002 2005 2008 2011 2013 Recession Recession Double-dip recession

14. Multi-layered picture of volunteering behaviour Biographical time In-depth volunteer analysis Behaviour and attitude analysis for volunteers within the population Focus on individuals increases Sample size decreases Chronological time Changes in social, political and moral attitudes over time Contextual time

15. Design versus practice • Longitudinal mixed-methods are more complicated than a single methodological approach • Over-estimation of mixed methods: It has not been possible to answer all of the designed research questions with the data chosen, the fit of the samples and the timing of the analysis • Did we achieve our mixed method dialogue? • Paradigm, background, and terminology differences make maintaining a mixed-method dialogue difficult • How time fits together: in practice time does not relate directly between different methodologies • Has the project benefited from using mixed-methods?

16. References • Bryman, C. (2008) ‘Why do Researchers Integrate/Mesh/Blend/Mix/Merge/Fuse Quantitative and Qualitative research?’, in M.M. Bergman (ed.) Advances in Mixed-Methods Research, London: Sage. pp 87-100. • Tashakkori, A. and Teddlie, C. B. (2008) Quality of Inferences in Mixed Methods Research: Calling for an Integrative Framework in in M.M. Bergman (ed.) Advances in Mixed-Methods Research, London: Sage. pp.101-119

17. Thank you for listening, any questions? Contact details: • R.Lindsey@soton.ac.uk • E.Metcalfe@soton.ac.uk Project website: http://longitudinalvolunteering.wordpress.com/

18. Back-up slides

19. The challenges of analysing secondary data over time Quantitative: • Variations in data collection process were difficult to uncover • The questions that were asked limits the data available • Data collected is set within the present time, only part of the life-course is recorded Qualitative: • Inconsistent descriptions of the life-course at different time-points • Lack of awareness of the what is happening within the present time • Accuracy of retrospective writings

20. Study design challenges The mixed-method design framed the study, and influenced how well the data sources fitted together. Compromises around the following choices needed to be made: • Choice of secondary data sources • Choice of timing of analyses • Choice of samples and how these substantively fit together • Choice of samples and how these fit together across time (thematic and temporal bunching)

21. Concurrent mixed method design Our research design aimed to potentially ‘offset’ the respective weaknesses of these two analytical methodologies by taking advantage of their joint strengths to provide a ‘complete[ness]’, and ‘comprehensive’ picture (Bryman, 2008, p.91) Qualitative data Project beginning Project end Quantitative data

22. Cross-sectional or Longitudinal?Synchronic or Diachronic? • The length of chronological time being researched affects our perceptions and understandings of behaviour and attitudes • Longitudinal/diachronic: following a person through time • Cross-sectional/synchronic: A certain point in time Hi, I’m Sarah Volunteered Children Retirement Marriage Do not volunteer 2008 2013 1984 1981 2011 1996 1999 2002 2005 1993 1990 1987 Increasing age Recession Recession Double-dip recession