Why do we need mixed-methods? Should we differentiate integration versus mixed-methods?

1. Why do we need mixed-methods?Should we differentiate integrationversus mixed-methods? Alan Bryman, Management Centre, University of Leicester, UK.

2. The current situation • Exciting times • ‘Paradigm wars’ • Epistemological arguments • Pragmatism • Washing machines and questionnaires! • Not addressing epistemological issues today

3. My Research on Multi-Strategy Research • Funded by Economic & Social Research Council – Research Methods Programme • 2 main strands today: • Traditional content analysis of journal articles using mixed-methods research • Interviews with social researchers who’ve used it

4. Research methods • Wanted to map general characteristics of mixed-methods research – content analysis • Wanted perspectives of mixed-methods researchers + contingencies involved – semi-structured interviews

5. Content analysis • Searched Social Sciences Citation Index for: quantitative and qualitative; triangulation; multi(-)method; mixed method in titles and abstracts of English language articles • 5 disciplines: sociology; social psychology; organizational behaviour; human and cultural geography; media and cultural studies • 1994-2003 • 232 articles

6. Content analysis • Articles foregrounded mixed-methods • Emphasis on mixed-methods in terms of data collection and analysis • Major focus on rationales for mixed-methods research

7. Classifying Forms of Mixed-Methods Research • Distinction between rationale and practice • Rationale = stated purpose(s) of integrating quantitative and qualitative research • Practice = actual use(s) made of integrating quantitative and qualitative research • Used both Greene et al. scheme and my own grounded scheme

8. Classifying Forms of Mixed-Methods Research • Greene et al. (triangulation; complementarity; development; initiation; expansion) • Parsimonious but only 2 rationales coded (primary & secondary) • Developed alternative scheme

9. Triangulation Offset Completeness Process Different research questions Explanation Unexpected results Instrument development Sampling Credibility Context Illustration Utility Confirm & discover Diversity of views Enhancement Other/unclear/not stated Alternative Scheme

10. Highlights of Findings: Rationale • No rationale in 27% of all articles • Main categories in terms of rationale: • Enhancement 32% • Completeness 13% • Sampling 13% • Triangulation 13%

11. Highlights of Findings: Practice • Main categories in terms of practice: • Enhancement 52% (rationale 32%) • Triangulation 35% (rationale 13%) • Completeness 29% (rationale 13%) • Illustration 23% (rationale 2%) • Sampling 19% (rationale 13%)

12. Highlights of Findings: Rationale and Practice • Rationale and practice not always in line • Rationale often not reflected in how multi-strategy research actually used • Practice often doesn’t chime with rationales given • Examples from contingency table analysis

13. Highlights of Findings: Rationale and Practice – The Case of Triangulation • Of the 29 articles citing triangulation as a rationale, 19 used it that way, i.e. one-third of articles citing triangulation as rationale didn’t use multi-strategy research that way or didn’t report doing so. Other prominent uses of articles citing triangulation were: enhance (13); completeness (10); and illustration (8).

14. Highlights of Findings: Rationale and Practice – The Case of Triangulation • Other way around • 80 articles used a triangulation approach but only 19 of them gave it as a rationale, i.e. three-quarters of articles using triangulation didn’t cite it as a rationale • Suggests triangulation hard to resist when opportunity arises

15. Highlights of Findings: Rationale and Practice – The Case of Completeness • Completeness was a rationale for 31 articles and 84% of them used it that way • But when practice is examined, 61% of all articles using a completeness approach didn’t specify it as a rationale

16. Highlights of Findings: Rationale and Practice – The Case of Enhancement • 73 articles specified enhance as rationale, a quarter of them didn’t use multi-strategy research this way • 121 articles used multi-strategy research this way, but over half of them hadn’t specified it as a rationale • Several other examples of mismatches

17. Highlights of Findings: Rationale and Practice • Often mismatch between rationale and practice • Mixed-methods research a moveable feast

18. Minority Strategy:The Gatling Gun Approach

19. Gatling Gun Strategy 4 or more rationales: • 6 articles in terms of rationale • 33 articles in terms of practice

20. Themes from Semi-Structured Interviews • Similar to content analysis • Mixed-methods research increasingly expected • Concern for many • Research questions important • Particularistic versus universalistic discourses • Not due to confusion – lack of guidelines about mixed-methods issues; textbook account too simple; ambivalence about role of research questions; lack of prescription •  Uncertainty

21. What do we mean by mixing, etc.? • Mixing vs. integration • Use of verbs • What does bringing together of quantitative and qualitative research entail? • Mixed-methods research or multi-methodology/multi-method research?

22. Is Integration Occurring?Content analysis findings • Content analysis findings • Genuine integration – 18% of articles • Parallel presentation – 47% of articles • Looked for evidence of findings being brought together  comprehensive picture interweaving both

23. Is Integration Occurring?Semi-structured interviews Most expressed concern. Main themes: • Different audiences • Greater faith in one; also familiarity • Design issues • Time-lines differ • Skill specialisms • One more striking or interesting • Objectivist vs. constructionist accounts • Journal publication issues

24. Bryman Goes Reflexive • Mine was a mixed-methods project • Justified using both content analysis and qualitative interviewing quite well • Outcomes consistent with rationales • Mixed-methods research linked to my research questions • Integration of data not adequately achieved

25. Back to the Title

26. Why do we need mixed-methods? • We often don’t need it • But difficult to decide when we do • Good deal of uncertainty about when to use a mixed-methods approach

27. Should we differentiate integration vs. mixed-methods? • Don’t get preoccupied with the verbs • Interviewees saw problem • Mixing findings like a conversation • Forging an overall account • Lack of templates

28. Should we differentiate integration vs. mixed-methods? • Tended to emphasize doing mixed-methods research • Need more attention to representation of mixed-methods findings in publications • Quality criteria