Qualitative Evidence Synthesis (QES):

1. QualitativeEvidenceSynthesis (QES): Karin HannesCentre forMethodology of Educational Research

2. Qualitative Evidence Synthesis: Agenda PART 1 • Its context (and how I becametriggeredby QES) • How to formulatequestions and search forqualitativeinsights PART 2 • Itspopularity • Itsrole in systematicreviews (practical examples) • A list of developedapproaches • A comparison of characteristics of twocommonlyusedapproaches PART 3 • New developments • Context specific versus multi-contextreviews • Mixed methodsreviews

3. PART 1: CONTEXT AND STEPWISE APPROACH

4. CONTEXT Emma, Born the 6th of October 2010

5. CONTEXT

6. CONTEXT • Women who exercised did not lose significantly more weight than women in the usual care group. • Women who took part in a diet or diet plus exercise programme, lost significantly more weight than women in the usual care. • There was no difference in the magnitude of weight loss between diet and diet plus exercise group. • The interventions seemed not to affect breastfeeding performance adversely.

7. CONTEXT A recent study in the Journal of the America College of Nutritianfoundthatthosewhoatecerealswherelower in weightcompared to thosewhoatemeat and eggs, breadorskippedbreakfast.

8. CONTEXT Logical, rational reasoning: • IF a diethelps to looseweightafterpregancy. • IF cereals have proven to workwell as a diet. • THEN the consumption of cerealswilllead to weight loss afterpregnancy! • Afterhavingconsumedcerealsforseveralmonths……………..

9. CONTEXT What is evidence? • Evidence of ‘effectiveness’: the extent to which an intervention, when used appropriately, achieves the intended effect. • Evidence of ‘feasibility’: the extent to which an intervention is practical and practicable, whether or not an intervention is physically, culturally or financially practical or possible within a given context. • Evidence of ‘appropriateness’ the extent to which an intervention fits with a situation, how an intervention relates to the context in which it is given. • Evidence of ‘meaningfulness’: the extent to which an intervention is positively experienced by the population and relates to the personal experience, opinions, values, beliefs and interpretations of the population.

10. CONTEXT SystematicReviews IF I amnotinterested in evidence of effectiveness, BUT in feasibility, appropriateness, meaningfulness,… QualitativeEvidenceSynthesis:A process of summarizingqualitative research findings, eitheraggregativeorinterpretive, bycomparing and analysingtextsderivedfrom multiple accounts of aneventorsituationas reported in basicqualitative research studies. Explore questions such as • how do people experience illness? • why does an intervention work (or not), for whom and in what circumstances…? • what are the barriers and facilitators to accessing health care? • what impact do specific barriers and facilitators have on people, their experiences and behavior?

11. Systematic Reviews Could I stilluse the methodologyoutlinedfor SR to answer these questions? Could I useit in the sameway? Questionformulation  PICO becomes SPI(C)E Searching  Sensitive (all-inclusive) versus Specific (Selective) Criticalappraisal  Proponents versus Opponents Synthesis  Variety of Approaches Recommendations  Notalways the goal, couldbe building theory as well CONTEXT

12. QES: Stepwise approachPICO becomes SPICE SPI(C)E ! I Intervention/(topic of) Interest: diet (C) Comparison: (training, placebo) S Setting: Western P Perspective: Mothers in a post-natalsituation For somequalitativequestionsthereisn’tanintervention to beevaluated. • EEvaluation: Attitude, view, opinionon... • Elementsthat hinder the diet • Impact of the dieton the generalwelbeing of the mother • Opinionsonhow to best integrate the diet in dailyfamilylife •  Targetedsuggestionsonhow to makethingswork.

13. QES: StepwiseapproachSearchingqualitativeevidence: problems • Littleresultfromsearching the major databases • 30% database & handsearch • 50% ‘snowballing’ • 24% personalcontacts (Greenhalgh, 2005) • Major problems: • Bad indexing • Lessdeveloped and testedmethodological filters • MeSH-term: qualitative research • General rules: • Usemethodological filters • Examinereferences • Use the relatedarticle features in major databases • Search forcitations (backword and forward)

14. QES: Stepwise approachSearching qualitative evidence: INTERTASCwww.york.ac.uk/inst/crd/intertasc/index.htm • nursing methodology research OR (qualitative OR ethnolog* OR ethnog* OR ethnomethodolog* OR emic OR etic OR phenomenolog*[Title/Abstract]) OR (hermeneutic* OR heidegger* OR husserl* OR colaizzi* OR giorgi* OR glaser OR strauss [title/abstract]) OR (kaam* OR manen OR participant observ* OR constant compar* [title/abstract]) OR (focus group* OR grounded theory OR "narrative analysis" OR lived experience* OR life experience* [title/abstract]) OR (theoretical sampl* OR purposive sampl* OR ricoeur OR spiegelberg* OR merleau [title/abstract]) OR (metasynthes* OR meta-synthes* OR metasummar* OR meta-summar* OR metastud* OR meta-stud* [title/abstract]) OR (maximum variation OR snowball [title/abstract]) OR (field stud* OR field note* OR fieldnote* OR field record* OR action research [title/abstract]) OR (thematic analys* OR content analy* OR unstructured categor* or structured categor* [title/abstract]) OR (participant observation* OR nonparticipant observation* OR non participant observation* [title/abstract]) OR (tape recording OR "tape record*" OR "video record*" OR "audio record*" OR taperecord* OR audiorecord* OR videotap* OR videorecord*)  Methodology Citations/authors Techniques http://www.library.ualberta.ca/subject/healthsciences/QualitativeFilters/index.cfm

15. QES: Stepwise approachSearching qualitative evidence: debate • We need a sensitive approach to searching that includes all potentially relevant studies. • We need a search strategy that is specific and purposeful, including studies that are relevant to our synthesis. This might include working with a saturation point for inclusion. •  Related to the goal and/or approach of a particular study. Searching: addressed in upcomingpresentation

16. The more you appraise the more you obstruct the creative, interpretive nature of qualitative research. The most important criterion is relevance! The more you appraise the smaller the chance that you would end up with statements and conclusions that do not represent the truth. The most important criterion is (methodological) quality! QES: Stepwise approachCritically appraising evidence: debate ‘Criticalappraisal’ addressed in upcomingpresentation

17. PART 2: Qualitative Evidence Synthesis (QES) approaches QES in the scientific literature The role of QES (in SR) A list of developed approaches Comparing characteristics of two commonly used approaches

18. QES in the public healthliterature Timetrend basedon the reviewsfrom: Dixon-Woods & colleagues (2007) Hannes & Macaitis (2011)

19. Meta-analysis Evidence-synthesis Nature Aim Studies Result Synthesis The role of QES in SR: different aims Qualitative Make sense of data Basic comparability Added value in content Through interpretation Quantitative Accumulating Strictly comparable More power Through data

20. The role of QES in SR • Inform:reviewsbyusingevidencefromqualitative research to help define and refine the question, and to ensure the reviewincludesappropriate studies and addresses important outcomes (scopingreview) • Enhance:reviewsbysynthesizingevidencefromqualitative research identifiedwhilstlookingforevidence of effectiveness (process and implementation issues). • Extend:reviewsbyundertaking a search to specificallyseek out evidencefromqualitative studies to addressquestionsdirectlyrelated to the effectivenessreview (mixed methodormultilevelsynthesis). • Supplement: reviewsbysynthesizingqualitativeevidencewithin a stand-alone, butcomplementaryreview to addressquestionsonotherthaneffectiveness (stand-aloneor parallel synthesis). Note:These shouldbedistinguishedfrom a narrative report of a quantitative SR. Whenindividual studies cannotbepooledquantitatively (reason: heterogeneity), theymaystill have usefulqualitativeinformation to besharedwith the reader.

21. Example:Narrative report Adding a bit won’twork. The need: full SR of QR ! Narrative synthesis because pooling is not possible? Rather not labelled as QES…

22. Example: Extending Review Which interventions match recommendations derived from children’s views and experiences? ‘Mixed MethodApproach’Children & HealthyEating - EPPI-centre: eppi.ioe.ac.uk

23. Example: Supplementing review • Barroso J, Powell-Cope GM. Metasynthesis of Qualitative Research on Living with HIV Infection. Qual Health Res vol 10, nr 3, 2000.: • Understandexperience of adults living with HIV infection • 21 articles • Method: constant comparativeanalysis • Themes: • Findingmeaning in HIV: death, loss, surviving • Shatteredmeaning: fear, irreparable • Humanconnectedness: meaningfullrelationshipsalienation, isolation • Focusingon the self: actions to enhancefysical and emotionalhealth • Negociating care: activerole • Dealingwith stigma: personal, family, society

24. The role of QES in SR • Provide evidenceon the subjectiveexperience of thoseinvolved in developing, delivering and receivinganintervention OR/AND • Provide a research-based context forinterpreting and explaining trial results • How to achievechange (more effectively)? • How to improveinterventions? • How to ‘fit’ subjectiveneeds? • Whatother type of interventionsmightbeneeded? OR/AND • Reveal the extent to whicheffectiveinterventions are actuallyadopted in policies and practice (what are barriers and bridges?) OR/AND • Contribute to the understanding of heterogeneity in outcomes

25. A list of developed approaches

26. A list of developed approaches • MAKING SENSE OF THE MYRIAD OF QUALITATIVE EVIDENCE SYNTHESIS METHODS • No clear guidance about how to determine which of the existing synthesis methods best fits a particular purpose. • possible considerations…

27. A list of developed approaches

28. Decision to conduct a qualitative evidence synthesis Purpose of the additional qualitative synthesis Primarily to integrate and interpret qualitative and quantitative evidence within a single approach or integrated model Can be used to develop explanatory theory To aggregate / summarise / integrate qualitative data toaddressspecificquestions in relation to a Cochrane interventionreview To interpret synthesised qualitative evidence and develop explanatory theory or models Meta-ethnography Thematic analysis with theory generation Grounded theory Realist review EPPI (mixed) approach (thematic) Narrative synthesis Thematic analysis without theory generation Meta-aggregation Product Aggregated findings from source papers Product Explanatory theory, analytical or conceptual framework or interpretative framework/ mechanism Qualitative Evidence Synthesisapproaches

29. Comparing characteristics of different approaches (developed for synthesis)

30. PART 3: Recent developments Context-specific versus multi-context reviews Mixed methods reviews

31. Context-specific versus multi-contextsyntheses Quantitative Review Qualitative Review

32. Context-specific versus multi-contextreviews

33. Context-specific versus multi-contextreviews • Context-specificsyntheses do well in responding to the needs and policies of a targeted setting. • Multi-contextsynthesesassist in building a cumulative knowledge base and are an excellent choicewhenlittleheterogeneityisexpected. • Integrating the best of both • Umbrellareviews, in which insights in a particularphenomenongeneratedfromdifferent settings couldbesummarized. • Transcontextual adaptation, whichmeansmodifying insights in such a waythattheybecome relevant and reply to the needs and policies of a targeted setting.

34. Mixed methodsreviews: definition •‘Mixed research synthesis’ (Sandelowski et al., 2006) •‘Mixed studies review’ (Pluye et al., 2009) •‘Mixed methods synthesis’ (Harden & Thomas, 2005) • ‘Mixed methods research synthesis’ = A synthesis in which researchers combine primary qualitative, quantitative, and mixed methods studies, and apply a mixed methods approach in order to integrate those studies, for the broad purposes of breadth and depth of understanding and corroboration (Heyvaert, Onghena & Maes, 2011). •Promising utility for research and practice: Combining strengths of qualitative and quantitative techniques and studies

35. Research Question(s) Cochrane Intervention Review Qualitative Evidence Synthesis Synthesis of Qualitative and Quantitative Studies Paired: Concurrent but separate review example – 1 team • Paired: A/S -QUAL + QUAN • –Equal status • -Concurrent • Lins S, Rücker G, Motschall E, Langer G, Antes G, Meyer G. Efficacy and experiences of telephone counselling for informal carers of people with dementia (Protocol). Cochrane Database of Systematic Reviews 2011, Issue 5. Art. No.: CD009126. DOI: 10.1002/14651858.CD009126 • Qualitative thematic synthesis related to the efficacy of telephone counselling • Leiknes KA, Berg RC, Smedslund G, Jarosch-von Schweder L, Øverland S, Hammerstrøm KT, Høie B. Electroconvulsive therapy for depression (Protocol). Cochrane Database of Systematic Reviews 2011, Issue 5. • Meta-aggregative approach to qualitative synthesis

36. Research Question(s) Cochrane Intervention Review Qualitative Evidence Synthesis Synthesis of Qualitative and Quantitative Studies Paired: Concurrent but separate review example – 2 teams • Embuldeniya, G. Et al. Perceived impacts and experiences of peer support in chronic disease • Qualitative meta-ethnography exploring issues related to an ongoing Cochrane effectiveness review (Doull MJ, O'Connor AM, Robinson VA, Tugwell P, Wells G. Peer support strategies for people with chronic disease to enhance health and promote health equity. Cochrane Database of Systematic Reviews 2005, Issue 3.) • Paired: A/S -QUAL + QUAN • –Equal status • -Concurrent

37. Retrospective, stand alone review example Retrospective, stand alone • -Equal status • -Sequential • Noyes, J., Popay, J (2007).Directly observed therapy and tuberculosis: how can a systematic review of qualitative research contribute to improving services? A qualitative meta-synthesis. Journal of Advanced Nursing, 57, 227-243. • Narrative Summary approach extending/enhancing a Cochrane intervention review Volmink J. & Garner P (1997) (updated 2002, 2006) Directlyobservedtherapyfortreatingtuberculosis. Cochrane Database SystematicReiews 19(2), CD003343. • Jordan J, Rose L, Dainty KN, Noyes J, Clarke S, Blackwood B. Factors which impact on the use of weaning protocols for reducing the duration of mechanical ventilation in adults and children. • Meta-ethnographyextending/enhancing a recent Cochrane intervention review (Blackwood B, Alderdice F, Burns KEA, Cardwell CR, Lavery G, O'Halloran P. (2010) Protocolized versus non-protocolized weaning for reducing the duration of mechanical ventilation in critically ill adult patients. Cochrane Database of Systematic Reviews (in press).

38. QES: conclusion • Ifyouwish to embark in the field of qualitativeevidencesynthesisyouwillsoonrealisethat • There are nofixedstandards • There is nogeneral consensus (and willthere ever beone?) • Youshall have to travel slowly, • Byyourownmeans(but the CQRM- and CP&I-groupscan help) • Onsmall and potentiallydifficultroads… • BE PREPARED FOR A POTENTIAL DELAY!