Integrative Mixed-Methods Research: A Workshop

1. P154 Integrative Mixed-Methods Research: A Workshop Theory, Methods and Applications Felipe González Castro, M.S.W., Ph.D., Professor Department of Psychology and Southwest Interdisciplinary Research Center Arizona State University, & Tanya Nieri, Ph.D., Assistant Professor Department of Sociology, University of California Riverside Presentation at the Hispanic Health Disparities Research Center, University of Texas at El Paso, August 14, 2009, El Paso, TX

2. I Introduction and Orientation

3. Purpose of This Workshop * The purpose of this workshop is to introduce and illustrate an Integrative Mixed-Methods(IMM) approach, and to allow participants to utilize this approach in their own research. * This includes: * Introducing conceptual, methodological, and applied issues * Understanding the strategy for integrating qualitative and quantitative evidence (data) * Discussing and illustrating data analytic and interpretive approaches * Describing and illustrating the IMM methodology including practice exercises

4. Overview of Workshop Objectives 1. To provide a framework for understanding the IMM approach 2. To review issues in the conceptualizationand measurement under the IMM approach 3. To examine research design and implementation issues 4. To offer interactive question-and-answer periods to discuss issues in the application of the IMM approach. 5. To present practice exercises in the use of the IMM methodology.

5. Strengths of Quantitative and Qualitative Approaches * Qualitative * Rich details of a phenomenon under study * Contextualization * Meaning and interpretation * Quantitative * Valid and reliable measurement * Assessment of strength of associations * Group comparisons * Hypothesis testing

6. Advantages of Mixed-Methods Approaches * Aims of general mixed-methods approaches are to obtain “the best of both worlds” by using both qualitative and quantitative procedures within a single study (Bryman, 2007). * Limitations of quantitative only studies * Measurement decontextualizes evidence * Results may be precise, but nevertheless lacking in explanatory power. Bryman, A. (2007). Barriers to integrating quantitative and qualitative research. Journal of Mixed Methods Research, 1, 8-22.

7. Limitations of Qualitative and Quantitative Approaches * Limitations of qualitative only studies * Difficulties in integration across cases * Cannot examine strength of associations among variables * Conclusions are suggestive, not confirmatory * Small sample sizes greatly limit generalizability and applicability. * Limitations of sequential mixed-methods designs * (Bryman, 2007) Bryman, A. (2007). Barriers to integrating quantitative and qualitative research. Journal of Mixed Methods Research, 1, 8-22.

8. Advantages of Mixed-Methods Approaches * Integrative Mixed-Methods aims to be a “best combination” design * Strength of confirmatory results derived from the quantitative analyses * Deep-structure explanatory descriptions from analysis of the qualitative text narratives * Offers a concurrentand integrative methodology for rigorous analyses that aim to attain, “the best of both methodologies: An integrative QUAL+QUANT design (Castro, Kellison, Boyd & Kopak, under review). Castro, F. G., Kellison, J., Boyd, S., Kopak, A. (under review). A methodology for conducting integrative mixed-methods research and data analyses.

9. Specific Gains from the IMM Approach * Allows greater authenticity in the study of culture, cultural nuances * Facilitates a “deep-structure” study of complex constructs * Acculturation: A complex process of culture change * Traditionalism: Core cultural orientations and values that persist across time * Cultural Gender Roles: Machismo, Marianismo * Cultural Identity: Mexican, Mexican American, Puerto Rican conceptions and feelings about that identity.

10. Specific Gains from the IMM Approach * Allows process analysis of developmental sequences * Short-term coping processes * Longer-term developmental effects * Allows analysis of complex emotional states * Ambivalence: The mixed emotion * Complex emotional states: Depression with anger vs. depression with anxiety

11. Specific Gains from the IMM Approach * Allows discovery of new aspects not previously identified * Can recontextualize results of quantitative analyses adding depth of meaning * Recontextualization “relates results of data analyses to their original context and lends depth of interpretation to the quantitatively derived results.” (Castro et al., under review; Castro & Coe, 2007). * Storylines: Allow deep-structure analysis of quantitative results by generating a descriptive story from the narratives of specific cases * The text narratives of contrasting groups of cases can used to develop storylines.

12. Discussion

13. II Theory and Conceptual Approach

14. Major Assumptions 1. Broad conception of measurement (“integrative qual-quant” approach): Two forms of research evidence 2. Focusing the method- Format of the focus question 3. Independence of observations/cases: A “Focus Group of One.” Unit of analysis is individual case. Creswell, J. W., Plano Clark, V. L., Gutmann M. L., & Hanson W. E. (2003). Advances in mixed methods research designs. In A. Tashakkori & C. Teddle (Eds.), Handbook of mixed methods in social and behavioral research (pp. 209-240). Thousand Oaks, CA: Sage.

15. Major Assumptions 4. Recurring themes exposed independently reflect consensual “cultural expressions” (beliefs, perceptions, attitudes, values) that exist within a local community 5. Rigor is essential in qualitative (and quantitative) design and analyses 6.“Concurrent Qual-Quant Design”allows transfer across evidence domains Creswell, J. W., Plano Clark, V. L., Gutmann M. L., & Hanson W. E. (2003). Advances in mixed methods research designs. In A. Tashakkori & C. Teddle (Eds.), Handbook of mixed methods in social and behavioral research (pp. 209-240). Thousand Oaks, CA: Sage.

16. Conceptual Framework for IMM Research Castro, F. G., & Coe, K. (2007). Traditions and alcohol use: A mixed-methods analysis. Cultural Diversity and Ethnic Minority Psychology13. 269-284. ,

17. Conceptual Framework for IMM Research 7. Parallelism between qualitative and quantitative * “The greater the quantitative-qualitative parallelism incorporated a-priori into a study design, the easier it will be to transform and transfer the textual and numeric forms of evidence.” (Castro et al., under review). 8. Unified conceptualization of information as research evidence with verbal and numeric forms Castro, F. G., Kellison, J., Boyd, S., Kopak, A. (under review). A methodology for conducting integrative mixed-methods research and data analyses.

18. Conceptual Framework for IMM Research * Focus on a core construct, e.g., traditionalism or machismo * Six stages of an IMM research study * Can examine relationships among variables, e.g., convergent and discriminant validity of constructs, both for those quantitatively measured (scales) and those qualitatively constructed (thematic variables) (Castro et al., under review). Castro, F. G., Kellison, J., Boyd, S., Kopak, A. (under review). A methodology for conducting integrative mixed-methods research and data analyses.

19. Limitations of the IMM Method * Newly generated thematic variables are sample specific * The quality of probing influences narrative production and depth * Face validity of the constructed thematic variables; are they valid beyond face validity, e.g., construct validity? * Focus question approach suited to some, but not all, research questions * IMM is an emerging and evolving methodology with some issues still being addressed and resolved.

20. Culture and Context

21. Deep Structure in Cultural Research * “Culture” is a complex entity with many conceptionsand definitions(Baldwin & Lindsey, 1994). * Scientific research that effectively addresses cultural issues must recognize the complexity of culture, and conduct a “deep structure” approach (Resnicow, Braithwaite, Ahluwalia, & Butler, 2000) to the study of cultural effects and cultural contexts. * How can we integrate quantitative and qualitative evidenceand methodsinto an integrative methodology that also constitutes sound science? * Baldwin, J.R., & Lindsley, S.L. (1994). Conceptualizations of culture. Tempe, AZ: Arizona State University. * Resnicow, K., Soler, R., Braithwaite, R. L., Ahluwalia, J. S. & Butler, J. (2000). Cultural sensitivity in substance use prevention. Journal of Community Psychology, 28, 271-290.

22. Aims in Deep Structure Cultural Research * To generate recurring thematic content derived inductively across cases, and from open-ended responses to one or more focus questions (group-related thematic categories). * In a “focus group of one,” each focus question introduces a topic area, e.g., “What is a real macho like? A real macho is: ________, which allows rich responses to that question. * Independent interviews (independent data points) allow the statistical analysis of inductively constructed thematic variables.

23. Explanatory Power and Process Analysis * Simple “cultural indicators” e.g., race/ethnicity (Hispanic/Latino, non-Hispanic white, Black) even in large sample epidemiologic studies may offer high “predictive power” (power to detect or relate) but often yields low “explanatory power.” * The meaningof a statistically-significant association may be limited.

24. Ecological Contexts * Context : 1. refers to “…surrounding conditions that can affect a specific process or outcome (Castro, Shaibi & Boehm-Smith, 2009, p. 91). 2. refers to “conditional effects… a health outcome is dependent on the particular level of a specific or controlling condition that affects the outcome.” 3. characteristics or conditions “can temper or even modify the meaning of [an] event.” Castro, F. G., Shaibi, G. Q., & Boehn-Smith, E. (2009). Ecodevelopmental contexts for preventing type 2 diabetes in Latino and other racial/ethnic minority populations. Journal of Behavioral Medicine, 32, 89-105.

25. Ecodevelopmental Model * Sociocultural contexts are “surrounding”macro-levelenvironmental influences that can influence or otherwise modify the meaning of amicro-level event (an event at the biological, personal, or familial levels).

26. An Ecodevelopmental Temporal Effects Model Castro, F. G., Shaibi, G. Q., & Boehn-Smith, E. (2009). Ecodevelopmental contexts for preventing type 2 diabetes in Latino and other racial/ethnic minority populations. Journal of Behavioral Medicine, 32, 89-105.

27. An Ecodevelopmental Temporal Effects Model Castro, F. G., Shaibi, G. Q., & Boehn-Smith, E. (2009). Ecodevelopmental contexts for preventing type 2 diabetes in Latino and other racial/ethnic minority populations. Journal of Behavioral Medicine, 32, 89-105.

28. Aims of the Integrative Mixed Methods Approach * In addition to generating specific confirmatory results, the integrative mixed-methods approach aims to generate results rich in explanatory power: A more complete explanation of: * A complex construct * A sociocultural process * Complex interaction effects Castro, F. G., Kellison, J., Boyd, S., Kopak, A. (under review). A methodology for conducting integrative mixed-methods research and data analyses.

29. Assumptions and Approach * The mixed-methods approach (Creswell et al, 2003; Hansen et al, 2005) can be more informative in identifying person x situation aspects of health (Creswell et al., 2003; Hansen et al., 2005). * As applied to health research, the mixed methods approach can uncover previously undetected thematic patterns that exist within special populations. * Creswell, J. W., Clark, V. L. P., Gutmann, M. L., & Hanson, W. E. (2003). Advanced mixed methods research designs. In A. Tashakkori & C. Teddle (Eds.), Handbook of mixed methods in social and behavioral research (pp. 209-241). Thousand Oaks, CA: Sage. * Hansen, W. E., Creswell, J. W., Clark, V. L. P., Petaka, K. S., & Creswell, J. D. (2005). Mixed methods research designs in counseling psychology. Journal of Counseling Psychology, 52, 224-235.

30. Starting with an Appreciation for Culture * Value of cultural factors as an important in scientific research, especially with special populations. * Devaluing cultural factors promotes culturally-shallow research. * Incorporating cultural factors from the beginning; at the stage of research conceptualization and design. * Need rigor in both the design and implementation of qualitativeand quantitativeaspects of the research study.

31. Machismo & Marianismo: Complex Cultural Constructs * Machismo - A traditional Latino gender role orientation that emphasizes male dominance and privilege as a proper form of male conduct. * Marianismo- A traditional Latino female role orientation that emphasizes motherly nurturance, and the demure and pure identity of a virgin (Virgin Mary) as a proper form of female conduct (Paniagua, 1998). Paniagua, F. A. (1998). Assessing and treating culturally diverse clients: A practical guide (2n ed). Thousand Oaks, CA: Sage.

32. Discussion

33. III The Integrative Mixed-Methods Approach

34. Design Guidelines * Parallelism in a unified conceptualization of research evidence as numeric and text narrative formats. * Explicitly integrative design; emphasizes the linkageand interchangeof qualitative and quantitative data (Bryman, 2007). * Purposive sampling; sample size ranging from 40 to 200. * Concurrent (QUAL+QUANT) design (not sequential). Bryman, A. (2007). Barriers to integrating quantitative and qualitative research. Journal of Mixed Methods Research, 1, 8-22.

35. Parallel Design Elements * Qualitative: Clear and specific focus questions aid in generating specific and informative narrative content that is obtained via individual open-ended interviews. * Quantitative: A sound choice of measured variables allows the measurement of complementary scaled variables. * This allows the conduct of convergent and discriminant validity (statistical triangulation) using planned correlational analyses.

36. Balancing Specificity with Rich Complexity * Quantitative measured variables (scales) generate specific scores but may be shallow in content. * Qualitative responses (thematic categories) may be rich in content, but may be non-specific. * The integrative mixed methods approach aims to attain the best of both approaches under a unified, rigorous and informativemethodology.

37. Rigor in Design and Implementation * Individual focus question interviews, and thus independence of observations/cases, allows statistical analysis of data. * Can evaluate the strength of association among measured variables (from scales) and among thematic variables (constructed from thematic text analysis). * Statistical triangulation can be conducted from correlational analyses of measured variables and thematic variables.

38. Steps in Conducting the IMM Approach

39. Six Steps in the IMM Approach 1. Create focus question and conduct interviews (COLLECTION) 2. Extract response codes from interview transcripts (CONVERSION) 3. Create thematic categories – “families” (CONVERSION) 4. Assign axial codes to create thematic variables (CONVERSION) 5. Conduct integrative data analysis (ANALYSIS) 6. Create storylines – recontextualize (INTERPRETATION)

40. Conceptual Framework for IMM Research Castro, F. G., & Coe, K. (2007). Traditions and alcohol use: A mixed-methods analysis. Cultural Diversity and Ethnic Minority Psychology13. 269-284. ,

41. Step 1: The Focus Quesstion

42. Focus Questions - Machismo Beliefs and Identity Who You Are and Where You are Going: PLATICA - HISPANIC LEADERS (Version 12a, ENGL 4-11-06) IDNo. ____________ Today’s Date: _____________ Start Time: ________ Please answer these questions honestly on how you feel and what you think. The best answers are those which tell exactly what you think and feel. III. Gender Identity 3a. Male and Female Traits. Within the Latino/Hispanic cultures and in other parts of the world, men are often described as being “manly” or “macho” and women are often described as being “feminine.” 1a. Please tell me what a real “macho” man is like (their traits or characteristics). Tell me more. 1b. How much do you identify with this form of being macho? 2. Now, please tell me what a woman is like who is very “feminine” or “motherly.” Tell me more.

43. Step 1. Focus Questions * Are brief, personalized, and prompt a specific and “focused” response. * Can be framed in a sentence completion format: “How do you think about (conceptualize) acculturation as it occurs among Latino populations? 1a. “Acculturation is: ___________. 1b. “Acculturation” is best studied by: _______. * Must be designed based on a theoretical orientation or conceptual framework, and must be clearly constructed to aid in generating interpretable responses.

44. Step 1. Focus Questions * A well constructed focus question introduces a topic area, and prompts clear and specific responses, while also allowing a diversity of responses whereby this, “allows the voices of the participants to be heard.”

45. Text Analysis Flow Chart

46. Overview of the Text Analysis Flow Chart 1. Response narratives( Ri ) * Participants’ complete or full responses to the focus question. 2. Response codes( Cj ) * Specific portion of the participants’ responses to the focus question. 3. Thematic categories - “Families” ( Fk ) * Sets of response codes that have a common meaning; coded as mentioned=1, not=0. 4. Thematic variables( Vl ) * Axial coded thematic categories that are converted into numeric data.

47. Some Points Regarding Transcription Transcription Procedures 1. Access the transcription template file. 2. Transcriptions: A. Do not type out the main interviewer probes. B. Do type the additional probing questions used by the interviewers. C. Use “P:” and “I:” to indicate whether the participant or interviewer are the speakers. D. Go for meaning over exact words and utterances, while still capturing the main message expressed by the participant. E. Notation you may need: a. If you can’t decipher a word or phrase at all, use: [unclear]. b. If you have an important comment or observation to make about an event occurring within the tape, please indicate your comment within brackets, for example: [The participant cries at this point]. 3. If you encounter problems or some notable issues, please add these comments to your weekly report of progress.

48. About Atlas.ti * Atlas.ti is a multi-purpose qualitative analysis program that, “helps you uncover the complex phenomena hidden in your data.” (Atlas.ti, 2009). * Analysis capabilities include: * Text search tool with search and retrieve functions * Interactive boxes allowing “drag and drop” * Automatic and interactive forms of coding * A “family manager” for creating and managing thematic categories (families) * A filter tool to select subsets of text information * Export capabilities to SPSS, Excel, and for other programs * Web link: http://www.atlasti.com/ http://www.atlasti.com/quicktour.html

49. Step 2: Response Codes

50. Step 2. Response Codes • * A response code is linked to the participant’s Case ID Number. • * [T599] A macho is a man who protects his family from harm and also provides for them. * A response code is the specific answer (a quotation excerpt) to the focus question, as contrasted with: * incidental comments or statements that do not directly answer the focus question * a label that summarizes the content of the response • * Each case can have zero, one, two, three or more response codes.