Issues and challenges around integrating qualitative and quantitative data

1. Issues and challenges around integrating qualitative and quantitative data Angela Harden, Professor of Community and Family Health InCQuuiRES, 8th September, 2011

2. INSTITUTE FOR HEALTH AND HUMAN DEVELOPMENT • A centre for research, innovation and community engagement to promote health and wellbeing. • 15 members of staff from a range of health and social science disciplines. • Includes collaborative programme of research with Newham University Hospital NHS Trust (NUHT) focused on women, children and young people

3. Overview of current SR projects

4. Acknowledgements • Colleagues at the EPPI-Centre, Social Science Research Unit, Institute of Education • Co-convenors of the Cochrane Qualitative Research Methods Group

5. Outline • Drivers and challenges of integration • Review of approaches and methods • Comparing methods • Future challenges and issues

6. Drivers for integration • Greater recognition of the value of qualitative research in evidence-based policy • ‘Empty’ reviews • Public perspectives and experiences • Systematic reviews of complex interventions • Issues of process and implementation • Extension of evidence-based health care to other areas of public policy • Growing tradition for integration in primary research (mixed methods research) • Dedicated research funding for methodological research • Establishment of dedicated methods groups (e.g. Cochrane Qualitative Methods Research Group)

8. Differences in Male Life Expectancy within a small area in London Travelling east from Westminster, every two tube stops represent over one year of life expectancy lost –Data revised to 2004-08 Male Life Expectancy 73.6 (CI 71.9-75.2) Canning Town Male Life Expectancy 78.5 (CI 75.5-81.6) Westminster Canary Wharf London Bridge River Thames Canada Water North Greenwich Bermondsey Waterloo Southwark Electoral wards just a few miles apart geographically have life expectancy spans varying by years. For instance, there are eight stops between Westminster and Canning Town on the Jubilee Line – so as one travels east, every two stops, on average, mark over a year of shortened lifespan. 1 London Underground Jubilee Line 1 Source: Analysis by London Health Observatory of ONS and GLA data for 2004-08. Diagram produced by Department of Health

9. Mega-events and urban regeneration • Examples of complex interventions with multiple anticipated outcomes • Regeneration - a set of activities to reverse economic, social and physical decline • Housing stock, service improvements, physical environment, creation of socially mixed population

10. McCartney et al. BMJ 2010; 340:c2369

11. McCarthy et al. (2010) conclude……. “There is little evidence that major multi-sport events held between 1978 and 2008 delivered health or socio-economic benefits for the population of the host country…..” “Future events such as the 2012 Olympic Games and Paralympic Games, or the 2014 Commonwealth Games, cannot be expected to automatically provide benefits. Until decision makers include robust, long term evaluations as part of their design and implementation of events, it is unclear how the costs of major multi-sport events can be justified in terms of benefits to the host population.”

12. Unansweredquestions • What happens during implementation? • How do the various actors involved experience mega-sporting events and the associated regeneration? • What are the pathways to impact? • Do variations in the above relate to outcomes?

13. Challenges of integration • New and evolving field • Scale of task • Paradigm wars • Lack of training and expertise • Few worked examples • Infrastructure in development

14. Approaches and methods (1) • Narrative summary • Thematicanalysis • Grounded theory • Meta-ethnography • Meta-study • Miles and Huberman's data analysis techniques • Contentanalysis • Case survey • Qualitative comparative analysis • Bayesian meta-analysis “Qualitising” 2004 “Quantitising”

15. Approaches and methods ( 2 ) Bayesian meta-analysis Narrative synthesis Critical Interpretive synthesis Integration of qualitative and quantitative research Meta-narrative mapping Mixed methods synthesis Realist synthesis

16. Origins of approaches and methods

17. Comparing approaches and methods

18. In focus: Mixed methods synthesis • Aim is to generate and test theory from diverse body of literature • Exhaustive search, review questions, inclusion criteria and quality assessment largely specified a priori • Each review typically has three syntheses: • Statistical meta-analysis • Thematic synthesis • Cross-study synthesis Recent examples Teenage pregnancy and social disadvantage (Harden et al., 2009) Knowledge and information needs of young people with epilepsy (Lewis et al., 2010) Behavioural interventions for weight management in pregnancy (Campbell et al., 2011) The effects of schools and school environment interventions (Bonell et al., 2011)

19. Mixed methods systematic reviews Working definition Combining the findings of ‘qualitative’ and ‘quantitative’ studies within a single systematic review, in order to address the same, overlapping or complementary review questions (Harden and Thomas, 2010)

20. Three ways in which reviews are mixed…. • The types of studies included and hence the type of findings to be synthesised (i.e. ‘qualitative/ textual and quantitative/numerical) • The types of synthesis method used (e.g. statistical meta-analysis and qualitative synthesis) • The mode of analysis: theory testing AND theory building

22. Review questions • What is known about the barriers to, and facilitators of, healthy eating amongst children? • Do interventions promote healthy eating amongst children? • What are children’s perspectives on healthy eating? • What are the implications of the above for intervention development?

23. REVIEW PROCESS Searching, screening and mapping Focus narrowed to ‘fruit &veg’ Synthesis 2: Qualitative studies (n=8) 1. Quality assessment 2. Data extraction 3. Thematic synthesis Synthesis 1: Trials (n=33) 1. Quality assessment 2. Data extraction 3. Statistical meta-analysis Synthesis 3: Trials and qualitative studies

24. Synthesis 3: Across studies

25. Synthesis 3: Across studies Increase (standardised portions per day) in vegetable intake across trials Little or no emphasis on health messages

26. In focus: Bayesian synthesis • Aim is to test theory • Findings from qualitative and quantitative research are ‘fused’ • Only three worked examples to date • Variation in weight given to the qualitative evidence • Synthesis product is a set of weighted factors associated with/predicting the phenomenon under review Recent examples Factors affecting the uptake of childhood immunisation (Roberts et al., 2002) Factors that influence adherence to HIV medication regimes (Vollis et al., 2009) (Crandell et al., 2011)

27. Factors that influence adherence to HIV medication (Crandell et al., 2011)

28. In focus: Critical Interpretive Synthesis • Aim is to generate theory from large and diverse body of literature • Literature itself is an object of scrutiny (critical) • Comprehensive search to identify sampling frame • Purposive and theoretical sampling • Analysis leads to generation of synthetic constructs and a synthesising argument Recent examples Access to health care for vulnerable groups (Dixon-Woods et al. 2006) Use of morphine to treat cancer related pain (Flemming, 2009) Nurses response to suicide and suicidal patients (Talseth and Gilje, 2011

29. Access to health care for vulnerable groups (Dixon-Woods et al. 2006) • Based on 119 papers • Interpretive qualitative analysis of diverse types of studies • Conceptual and methodological problems with measures of health service utilisation • Synthesising argument organised around a set of central concepts (e.g. navigation, adjudications) with the synthetic construct of ‘candidacy’ at the core

30. Access to health care for vulnerable groups (Dixon-Woods et al. 2006) Core construct – candidacy “Candidacy describes the ways in which people's eligibility for medical attention and intervention is jointly negotiated between individuals and health services…….candidacy is a dynamic and contingent process, constantly being defined and redefined through interactions between individuals and professionals, including how "cases" are constructed.” Central concepts • Identification of candidacy • Navigation • Permeability of services • Appearances at health services • Adjudications • Offers and resistance • Operating conditions and local production of candidacy

31. In focus: Meta-narrative review • Aim is to make sense of and understand diverse bodies of literature and their findings • Literature itself is an object of scrutiny (critical) • Searching is iterative, ‘snowballing’ a key technique • Analysis leads to production of a set of meta-narratives (‘storylines of research’) Recent examples Spread and sustainability of innovations in health service delivery and organisation (Greenhalgh et al., 2005) Understanding the use of electronic patient records in health care organisations (Greenhalgh et al., 2009) See also the: Realist and meta-narrative evidence synthesis evolving standards project (RAMASES) (Greenhalgh et al., 2011)

32. A meta-narrative approach • The influence of Kuhn’s ‘paradigms’ (1962) and the makings of the first meta-narrative • The essential technique is interpretive synthesis exploring distinct research traditions, each with its own meta-narrative • Methods of ‘unpacking’ the meta-narrative: exploratory methods; expert consultations; snowballing; database searching

33. Stages of a meta-narrative review (from Greenhalgh et al., 2009)

34. Meta-narratives identified in the electronic patient records review (from Greenhalgh et al., 2009)

35. Comparing approaches and methods

36. Future challenges • More worked examples are key • Focus on methods and tools for the actual integration • Enhancing transparency • Establishing rigour • Further conceptual work to illuminate points of difference, strengths and weaknesses, fit for purpose • Learning from, and contributing to, the mixed methods literature for primary research.

37. Thank you!a.harden@uel.ac.uk Forthcoming book Gough D, Oliver S, Thomas J (Eds) (forthcoming, Feb 2012) An Introduction to Systematic Reviews. London: Sage