Qualitative Evidence Synthesis (QES):

1. QualitativeEvidenceSynthesis (QES): Karin HannesCentre forMethodology of Educational Research

2. Qualitative Evidence Synthesis: Agenda PART 1 • Its context (and how I becametriggeredby QES) • How to formulatequestions and search forqualitativeinsights PART 2 • Itspopularity • Itsrole in systematicreviews (practical examples) • A list of developedapproaches • A comparison of characteristicsof twocommonlyusedapproaches PART 3 • New developments • Context specific versus multi-contextreviews • Mixed methodsreviews

3. PART 1: CONTEXT AND STEPWISE APPROACH

4. CONTEXT Emma, Born the 6th of October 2010

5. CONTEXT

6. CONTEXT • Women who exercised did not lose significantly more weight than women in the usual care group. • Women who took part in a diet or diet plus exercise programme, lost significantly more weight than women in the usual care. • There was no difference in the magnitude of weight loss between diet and diet plus exercise group. • The interventions seemed not to affect breastfeeding performance adversely.

7. CONTEXT A recent study in the Journal of the America College of Nutritianfoundthatthosewhoatecerealswherelower in weightcompared to thosewhoatemeat and eggsorskippedbreakfast.

8. CONTEXT Logical, rational reasoning: • IF a diethelps to looseweightafterpregancy. • IF cereals have proven to workwell as a diet. • THEN the consumption of cerealswilllead to weight loss afterpregnancy! • Afterhavingconsumedcerealsforseveralmonths…………….. • Ifit has proven itself to workforothers… • THEN WHY DOESN’T IT WORK FOR ME!

9. CONTEXT What is evidence? • Evidence of ‘effectiveness’: the extent to which an intervention, when used appropriately, achieves the intended effect. • Evidence of ‘feasibility’: the extent to which an intervention is practical and practicable, whether or not an intervention is physically, culturally or financially practical or possible within a given context. • Evidence of ‘appropriateness’ the extent to which an intervention fits with a situation, how an intervention relates to the context in which it is given. • Evidence of ‘meaningfulness’: the extent to which an intervention is positively experienced by the population and relates to the personal experience, opinions, values, beliefs and interpretations of the population.

10. CONTEXT SystematicReviews IF I amnotinterested in evidence of effectiveness, BUT in feasibility, appropriateness, meaningfulness,… QualitativeEvidenceSynthesis:A process of summarizingqualitative research findings, eitheraggregativeorinterpretive, bycomparing and analysingtextsderivedfrom multiple accounts of aneventorsituationas reported in basicqualitative research studies. Explore questions such as • how do people experience illness? • why does an intervention work (or not), for whom and in what circumstances…? • what are the barriers and facilitators to accessing health care? • what impact do specific barriers and facilitators have on people, their experiences and behavior?

11. Systematic Reviews CouldI stilluse the methodologyoutlinedfor SR to answer these questions? Could I useit in the sameway? Questionformulation  PICO becomes SPI(C)E Searching  Sensitive (all-inclusive) versus Specific (Selective) Criticalappraisal  Proponents versus Opponents Synthesis  Variety of Approaches Recommendations  Notalways the goal, couldbe building theory as well CONTEXT

12. QES: Stepwise approachPICO becomes SPICE SPI(C)E ! I Intervention/(topic of) Interest: diet (C) Comparison: (training, placebo) S Setting: Western P Perspective: Mothers in a post-natalsituation For somequalitativequestionsthereisn’tanintervention to beevaluated. • EEvaluation: Attitude, view, opinionon... • Elementsthat hinder the diet • Impact of the dieton the generalwelbeing of the mother • Opinionsonhow to best integrate the diet in dailyfamilylife

13. QES: StepwiseapproachSearchingqualitativeevidence: problems • Littleresultfromsearching the major databases • 30% database & handsearch • 50% ‘snowballing’ • 24% personalcontacts (Greenhalgh, 2005) • Major problems: • Bad indexing • Lessdeveloped and testedmethodological filters • MeSH-term: qualitative research • General rules: • Usemethodological filters • Examinereferences • Use the relatedarticle features in major databases • Search forcitations (backword and forward)

14. QES: StepwiseapproachSearchingqualitativeevidence: keywords • Technique: focus group, interview, action research, content analysis, thematic analysis... • Software: Nudist OR Nvivo OR Atlas-ti • Citations : Glaser and Strauss, Miles and Huberman,... • Phenomenon: perceptions, attitudes, user views, standpoints, viewpoints, experiences,... • Methodology: ethnographic, phenomenologic,… • Type of data: stories, narratives, descriptions, themes, exploration, findings(<--> results)

15. QES: Stepwise approachSearching qualitative evidence: INTERTASCwww.york.ac.uk/inst/crd/intertasc/index.htm • nursing methodology research OR (qualitative OR ethnolog* OR ethnog* OR ethnomethodolog* OR emic OR etic OR phenomenolog*[Title/Abstract]) OR (hermeneutic* OR heidegger* OR husserl* OR colaizzi* OR giorgi* OR glaser OR strauss [title/abstract]) OR (kaam* OR manen OR participant observ* OR constant compar* [title/abstract]) OR (focus group* OR grounded theory OR "narrative analysis" OR lived experience* OR life experience* [title/abstract]) OR (theoretical sampl* OR purposive sampl* OR ricoeur OR spiegelberg* OR merleau [title/abstract]) OR (metasynthes* OR meta-synthes* OR metasummar* OR meta-summar* OR metastud* OR meta-stud* [title/abstract]) OR (maximum variation OR snowball [title/abstract]) OR (field stud* OR field note* OR fieldnote* OR field record* OR action research [title/abstract]) OR (thematic analys* OR content analy* OR unstructured categor* or structured categor* [title/abstract]) OR (participant observation* OR nonparticipant observation* OR non participant observation* [title/abstract]) OR (tape recording OR "tape record*" OR "video record*" OR "audio record*" OR taperecord* OR audiorecord* OR videotap* OR videorecord*)  Methodology Citations/authors Techniques http://www.library.ualberta.ca/subject/healthsciences/QualitativeFilters/index.cfm

16. QES: Stepwise approachSearching qualitative evidence: debate • We need a sensitive approach to searching that includes all potentially relevant studies. • We need a search strategy that is specific and purposeful, including studies that are relevant to our synthesis. This might include working with a saturation point for inclusion. •  Related to the goal and/or approach of a particular study. Searching: addressed in upcomingpresentation Next step ‘criticalappraisal’ addressed in upcomingpresentation

17. PART 2: Qualitative Evidence Synthesis (QES) approaches QES in the public health literature The role of QES (in SR) A list of developed approaches Comparing characteristics of different approaches

18. QES in the public healthliterature Timetrend basedon the reviewsfrom: Dixon-Woods & colleagues (2007) Hannes & Macaitis (2011)

19. Meta-analysis Evidence-synthesis Nature Aim Studies Result Synthesis The role of QES in SR: different aims Qualitative Make sense of data Basic comparability Added value in content Through interpretation Quantitative Accumulating Strictly comparable More power Through data

20. The role of QES in SR • Inform:reviewsbyusingevidencefromqualitative research to help define and refine the question, and to ensure the reviewincludesappropriate studies and addresses important outcomes (scopingreview) • Enhance:reviewsbysynthesizingevidencefromqualitative research identifiedwhilstlookingforevidence of effectiveness (process and implementation issues). • Extend:reviewsbyundertaking a search to specificallyseek out evidencefromqualitative studies to addressquestionsdirectlyrelated to the effectivenessreview (mixed methodormultilevelsynthesis). • Supplement: reviewsbysynthesizingqualitativeevidencewithin a stand-alone, butcomplementaryreview to addressquestionsonotherthaneffectiveness (stand-aloneor parallel synthesis). Note:These shouldbedistinguishedfrom a narrative report of a quantitative SR. Whenindividual studies cannotbepooledquantitatively (reason: heterogeneity), theymaystill have usefulqualitativeinformation to besharedwith the reader.

21. Example:Narrative report Adding a bit won’twork. The need: full SR of QR ! Narrative synthesis because pooling is not possible? Rather not labelled as QES…

22. Example: Extending Review Which interventions match recommendations derived from children’s views and experiences? Conclusion ‘Mixed MethodApproach’Children & HealthyEating - EPPI-centre: eppi.ioe.ac.uk

23. Example: Supplementing review • Barroso J, Powell-Cope GM. Metasynthesis of Qualitative Research on Living with HIV Infection. Qual Health Res vol 10, nr 3, 2000.: • Understand experience of adults living with HIV infection • 21 articles • Method: constant comparative analysis • Themes: • Finding meaning in HIV: death, loss, surviving • Shattered meaning: fear, irreparable • Human connectedness: meaningfull relationships alienation, isolation • Focusing on the self: actions to enhance fysical and emotional health • Negociating care: active role • Dealing with stigma: personal, family, society

24. The role of QES in SR • Provide evidenceon the subjectiveexperience of thoseinvolved in developing, delivering and receivinganintervention OR/AND • Provide a research-based context forinterpreting and explaining trial results • How to achievechange (more effectively)? • How to improveinterventions? • How to ‘fit’ subjectiveneeds? • Whatother type of interventionsmightbeneeded? OR/AND • Reveal the extent to whicheffectiveinterventions are actuallyadopted in policies and practice (what are barriers and bridges?) OR/AND • Contribute to the understanding of heterogeneity in outcomes

25. A list of developed approaches

26. A list of developed approaches • MAKING SENSE OF THE MYRIAD OF QUALITATIVE EVIDENCE SYNTHESIS METHODS • No clear guidance about how to determine which of the existing synthesis methods best fits a particular purpose. • possible considerations…

27. A list of developed approaches

28. Decision to conduct a qualitative evidence synthesis Purpose of the additional qualitative synthesis Primarily to integrate and interpret qualitative and quantitative evidence within a single approach or integrated model Can be used to develop explanatory theory To aggregate / summarise / integrate qualitative data toaddressspecificquestions in relation to a Cochrane interventionreview To interpret synthesised qualitative evidence and develop explanatory theory or models Meta-ethnography Thematic analysis with theory generation Grounded theory Realist review EPPI approach (thematic) Narrative synthesis Thematic analysis without theory generation Meta-aggregation Product Aggregated findings from source papers Product Explanatory theory, analytical or conceptual framework or interpretative framework/ mechanism Qualitative Evidence Synthesisapproaches

29. Comparing characteristics of different approaches (developed for synthesis)

30. PART 3: Revent developments Context-specific versus multi-context reviews Mixed methods reviews

31. Context-specific versus multi-contextsyntheses Quantitative Review Qualitative Review

32. Context-specific versus multi-contextreviews

33. Context-specific versus multi-contextreviews • Context-specificsyntheses do well in responding to the needs and policies of a targeted setting. • Multi-contextsynthesesassist in building a cumulative knowledge base and are an excellent choicewhenlittleheterogeneityisexpected. • Integrating the best of both • Umbrellareviews, in which insights in a particularphenomenongeneratedfromdifferent settings couldbesummarized. • Transcontextual adaptation, whichmeansmodifying insights in such a waythattheybecome relevant and reply to the needs and policies of a targeted setting.

35. QES: conclusion • Ifyouwish to embark in the field of qualitativeevidencesynthesisyouwillsoonrealisethat • There are nofixedstandards • There is nogeneral consensus (and willthere ever beone?) • Youshall have to travel slowly, • Byyourownmeans (but the CQRM-groupcan help), • Onsmall and potentiallydifficultroads… • BE PREPARED FOR A POTENTIAL DELAY!