Professor Jane Noyes

1. Qualitative Evidence SynthesisWhere are we?What’s New and What’s nextand outstanding challenges? Professor Jane Noyes Noreen Edwards Chair in Health Services Research and Child Health, Bangor University Cochrane Fellow and Lead Convenor Cochrane Qualitative Research Methods Group Editor Journal of Advanced Nursing

2. Most methods of primary qualitative data analysis can be applied to qualitative evidence synthesis ‘New’ evidence synthesis approaches have emerged –eg meta-ethnography – but use recognisable qualitative analysis principles There are many methods with similar sounding names – rush to publish ‘my method’ and call it something slightly different – meta-narrative, meta-ethnography, meta-summary, meta-synthesis, meta-aggregation – etc etc. Some reviewers are mixing or combining methods – eg thematic and realist into a single review What have we learned so far?

3. Many authors appear to stick with familiar methods rather than select the most appropriate method to address the question and type of evidence Software evolving and improving: EPPI reviewer / Atlas Ti Searching methods and approaches have evolved for specific methods of qualitative synthesis (purposive versus exhaustive, tipping point) There are increasing numbers of some very good and inevitably some very bad reviews published! Cochrane is finally getting to grips with qualitative evidence synthesis! What have we learned so far cont?

4. Huge increases in qualitative syntheses published • There is a big gap between what people claim to use as a synthesis approach and what is actually done in practice • Do not need more methods – but better application and further evaluation of existing ones

5. Aggregated themes versus theory development versus– could we do better? Karin is particularly critical of the way meta-ethnography is reported in published reviews – line of argument synthesis Like others my experience of theory development is that it takes months and months of thinking, meeting, discussing, and ‘arguing’! Some health and policy funders eg NICE require a product within 90 days RAPID QUALITATIVE EVIDENCE SYNTHESIS METHODOLOGY

6. New and updated guidance and new acceptance of the value of qualitative evidence

7. Guidance on new applications

8. Thematic synthesis –various types with different starting points - inductive and deductive 3 stage thematic synthesis - Line by line inductive coding- eg Thomas and Harden A priori - eg 5 stage Ritchie and Spencer Framework Synthesis Conceptual frameworks and models to guide analysis and synthesis Additional adaptations to conceptual frameworks to ‘best-fit’ specific reviews – eg conceptual development by the back door Greater specification and further adaptation of methods for qualitative evidence synthesis

10. Richie and Spencer Framework Synthesis Approach With Normalisation Process Theory Elements (Watson et al 2011)

11. ‘Best Fit Framework Synthesis’ Carroll, C., Booth, A., Leaviss, J., & Rick, J. (2013). “Best fit” framework synthesis: refining the method. BMC medical research methodology, 13(1), 1-16.

12. Development of typologies of methods and greater classification of terminology A new typology put forward by Gough, Thomas and Oliver 2012

14. Methods for the synthesis of qualitative research: a critical review. Barnett-Page E, Thomas J.

15. Much greater emphasis on qualitative evidence synthesis in the context of complex interventions • Therapist dependent interventions (where the intervention is a combination of the therapist effect and the therapy or procedure and the effectiveness is potentially dependent on both); • Complex healthcare interventions (where the intervention is a combination of several actions, e.g. multidisciplinary health care in stroke units); multilevel public health interventions (e.g., a healthy living initiative that aims to impact behaviour at the community, school, and individual levels); • Professional or patient education interventions (e.g., introduction of clinical guidelines). • Complex interventions may contain a mix of effective, ineffective, and even harmful actions which may interact synergistically or dysynergistically or be interdependent.

16. Much greater emphasis on qualitative evidence synthesis in the context of complex interventions • Therapist dependent interventions (where the intervention is a combination of the therapist effect and the therapy or procedure and the effectiveness is potentially dependent on both); • Complex healthcare interventions (where the intervention is a combination of several actions, e.g. multidisciplinary health care in stroke units); multilevel public health interventions (e.g., a healthy living initiative that aims to impact behaviour at the community, school, and individual levels); • Professional or patient education interventions (e.g., introduction of clinical guidelines). • Complex interventions may contain a mix of effective, ineffective, and even harmful actions which may interact synergistically or dysynergistically or be interdependent.

17. Methodological Investigation of Cochrane Reviews of Complex Interventions (MICCI) Acknowledge fellow methodologists who met at Montebello January 2012

18. Applying qualitative methods to explore complexity

19. Development of little used methods • Qualitative Comparative Analysis (QCA); (actually a mixed method approach) is a promising method for providing evidence in situations where interventions interact with contexts, enabling causal pathways to be discerned from how sets of conditions combine with particular outcomes. • David Byrne (leading proponent of QCA) and Bridget Candy & Jackie Chandler (PhD students) • Byrn D. Evaluating complex social interventions in a complex world 2013 • Blackman et al (2013) Using Qualitative Comparative Analysis to understand complex policy problems • Candy, B., King, M., Jones, L., & Oliver, S. (2013). Using qualitative evidence on patients' views to help understand variation in effectiveness of complex interventions: a qualitative comparative analysis. 

20. Development of little used methods • Use of content analysis to conduct knowledge-building and theory-generating qualitative systematic reviews Finfgeld-Connett2013 • Systematically reviewing and synthesizing evidence from conversation analytic and related discursive research to inform healthcare communication practice and policy: an illustrated guide Parry and Land 2013 • Systematic text condensation: A strategy for qualitative analysis Malterud 2012 The procedure consists of the following steps: 1) total impression – from chaos to themes; 2) identifying and sorting meaning units – from themes to codes; 3) condensation – from code to meaning; 4) synthesizing – from condensation to descriptions and concepts.

21. Greater use of Conceptual Frameworks, Theoretical and Logic models • Programme logic is the way in which a ‘programme’ fits together, usually in a simple sequence of inputs, activities, outputs, and outcomes. • Programme theory goes a step further and attempts to build an explanatory account of how the intervention/programme/service works, with whom, and under what circumstances.

22. Mechanisms • MECHANISMS • Programme theory can be expressed as an elaborated programme logic model, where the emphasis is on causal explanation using the idea of ‘‘mechanisms’’ that are at work. • Mechanisms ‘occur’ between the delivery of the intervention/programme/ service and the occurrence of outcomes of interest. • Mechanisms are participants’ responses to the intervention/programme/ service. • The mechanism of change is not the intervention/programme/service per se but the response that the activities generate (iethe human behaviour – that can be largely captured by qualitative research ) Astbury 2010

23. “I think you should be more explicit here in Step Two.”

24. Use of logic models Anderson et al. 2011 Using logic models to capture complexity in systematic review Turley et al 2013 Slum upgrading review: methodological challenges that arise in systematic reviews of complex interventions

25. Health belief model: public information to prevent skin cancer

26. Multiple methods within a single HTA review

27. New publication standards Enhancing transparency in reporting the synthesis of qualitative research: ENTREQ BMC Medical Research Methodology, Vol. 12, No. 1. (27 November 2012), 181, doi:10.1186/1471-2288-12-181 by Allison Tong, Kate Flemming, Elizabeth McInnes, Sandy Oliver, Jonathan Craig

28. Most downloaded article JCE

29. Continuing challenges and issues • Many methods but a relatively small number consistently used • Need more high quality published examples and methodological testing • Evidence synthesis is cheaper than primary research – funders increasingly fund systematic reviews – need more teams with quan/qual skills • Need greater specification of methods for implementation reviews

30. Thank you ! Jane.noyes@bangor.ac.uk