Mixed Research Methods

1. Mixed Research Methods Debra D. Roberts Department of Psychology HOWARD UNIVERSITY

2. SCIENTIFIC METHOD • Process involving a systematic approach to assessing observed (behavior) through the use of empiricalevidence • Steps • Identify a Problem • Develop a Hypothesis or Research Question • Gather Empirical Evidence • Analyze Data • Disseminate Findings

3. Asian Americans are frequently deployed as racial mascots by pundits who fixate on their extraordinary levels of educational attainment. They comprise only 5.5% of the U.S. population, yet about one fifth of the entering classes in Ivy League universities like Harvard, Yale, and Princeton. Pundits have attributed these educational outcomes to cultural factors, underpinned by values or traits that are innately Asian. However, this cultural explanation fails to consider the pivotal role of U.S. immigration law which has ushered in a new stream of highly educated, highly skilled Asian immigrants. Hyper-selectivity (as opposed to hypo-selectivity) of contemporary immigration significantly influences the educational trajectories and outcomes in the members of the 1.5 and second generation beyond individual family or parental socioeconomic characteristics, leading to group-based advantages (or disadvantages) that are consequential. Analysis of qualitative data shows that the children of hyper-selected immigrant groups begin their quest to get ahead from more favorable starting points, are guided by a more constricting success frame, and have greater access to ethnic capital than those of other immigrant groups. In turn, hyper-selectivity gives rise to stereotype promise —the boost in performance that comes with being favorably perceived and treated as smart, high-achieving, hardworking, and deserving students—that benefits members of the group so stereotyped. Our analysis also suggests that, while the so-called positive stereotype enhances the academic performance of Asian American students, the same stereotype reproduces new stereotypes that hinder them as they pursue leadership positions in the workplace. We suggest that Asian American professionals face a bamboo ceiling—an invisible barrier that impedes their upward mobility much like the glass ceiling does for women. 

4. COLLECT DATA • Identify the BEST research design to fit the question • QUALITATIVE vs. QUANTITATIVE • Qualitative Research aims to complete a detailed description • Quantitative Research aims to classify features, count them, and construct statistical models in an attempt to explain what is observed

5. QUALITATIVE Researcher may only know roughly in advance what he/she is looking for Researcher is the data gathering instrument Data are in the form of words, pictures or objects Dataset is more “rich”, time consuming, and less able to be generalized Researcher tends to become subjectively immersed in subject matter QUANTITATIVE Researcher knows clearly in advance what he/she is looking for Researcher uses tools (questionnaires) to collect numerical data Data are in the form of numbers and statistics Dataset is more efficient, but may miss contextual detail Researcher tends to remain objectively separated from the subject matter THE DEBATE

6. EXPERIMENTAL METHODS • Involves the manipulation (or observation) of variables • Variable: Any event, situation, or behavior that has at least two values • Variables can be classified in a number of ways

7. CATEGORIES OF VARIABLES • Situational Variables • Describe characteristics of a situation or environment • Response Variables • Responses or behaviors of individuals • Participant Variables • Characteristics of the individual • Mediating Variables • Processes that mediate the effects of a situational variable or particular response

8. TYPES OF VARIABLES • Independent (IV) • Manipulated by the researcher • Considered to be the “cause” • Dependent (DV) • Observed change as a result of manipulated IV • Confounding (third variable factor) • Unintentionally contributed to the observed change in the DV

9. OPERATIONAL DEFINITION • REMEMBER: Research is about making observations of human behavior, attitudes, etc. in a systematic manner • VARIABLES are simply abstract concepts that must be translated into concrete forms of observation • OPERATIONAL DEFINITION: Describing the variable in terms of the operations or techniques the researcher uses to measure or manipulate it

10. QUANTIFYING CONSTRUCTS • We often utilize scales of measurement • Nominal • Values of the scale have no 'numeric' meaning • Ordinal • Scale assignment is by the property of "greater than," "equal to," or "less than." • Interval • Intervals between values are equal • Ratio • There is an absolute zero point for the scale

11. A FEW EXAMPLES • Researcher A wants to study the effects of exposure to different types of music on anxiety among college students • IV = Type of music you listen to • Operationalized as: Choice between genres of music (classical, reggae, hip hop, jazz, etc.) • DV = Anxiety • Operationalized as: Standardized measure of anxiety

12. EXAMPLE 2 • Researcher B notices a clear difference in prevalence of asthma symptoms between those children living in impoverished neighborhoods and those living in more affluent neighborhoods. She sets up a study as follows: • IV = Neighborhood • Group 1: Impoverished as defined by census data or median income • Group 2: Affluent as defined by census data or median income • DV = Prevalence of Asthma Symptoms • Use frequency checklist to be completed by parents

13. EXAMPLE 3 • After Hurricane Katrina, researcher C noticed a marked increase in the number of parents reporting that their child was experiencing night terrors. He wants to test the hypothesis that trauma causes night terrors: • IV = Level of Trauma • Group 1: Witnessed the hurricane but family not displaced • Group 2: Witnessed the hurricane and family displaced • Group 3: Did not witness the hurricane • DV = Night terrors • Experience night terrors post-hurricane (yes OR no)

14. NON-EXPERIMENTAL VS. EXPERIMENTAL • There are advantages and disadvantages to both methods • Experimental methods offer more “researcher” control • Experimental methods allow for the inference of “causation”

15. THREATS TO VALIDITY • Subject Loss (selective) • History • Maturation • Demand Characteristics • Experimenter Effects • Testing

16. “TRUE” EXPERIMENTAL DESIGN • Involves the direct manipulation of the variable(s) • Subjects must be assigned to groups • Randomization ensures that extraneous variables are taken into account Independent Groups Design • Experimental Group • Receives the treatment • Control Group • Receives no treatment (or placebo)

17. QUASI-EXPERIMENTAL DESIGN • Provides an important alternative when true experiments are not possible (usually occur in natural settings) • Lacks the degree of control found in true experiments • Consider threats to internal validity

18. PRETEST-POSTTEST • Step 1: Observe (O1) outcome/dependent variable among group members (baseline measure) • Step 2: Administer treatment (X) • Step 3: Observe outcome (O2) variable post treatment administration O1 X O2

19. NONEQUIVALENT CONTROL GROUP DESIGN • Better able to make causal claims by adding a comparison group to a simple pretest-posttest design • Observations are made at time 1 and time 2 for both groups O1 X O2 ------------------ O1 O2

20. COMPLEX DESIGNS • Studying the effects of two or more IVs in one experiment • Main effects vs. Interaction Effects • Describe design: • Identify number of IVs and number of levels for each IV • Class Examples…

21. INTERACTIVE EXAMPLES

22. DATA COLLECTION / ANALYSES • Decide what your units of analyses are • Operationalize your variables • Decide what your data collection method is • Survey, Interview (focus group), Instruments • Be mindful of your analytical procedures when collecting data • Measurement determines types of analyses

23. ANALYSES • Descriptive Statistics: Used to answer the question “What happened in the experiment?” rather than “Why it happened.” • Inferential Statistics: We want to know whether the IV has a reliable effect on the DV • Usually Alpha (α) set at .05

24. QUALITATIVE APPROACHES • PHENOMENOLOGICAL • ETHNOGRAPHIC • GROUNDED THEORY • ENDOGENOUS

25. PHENOMENOLOGY • PURPOSE: To help researchers understand participants’ point of view • PROCESS : Select topic that is personally meaningful (emotionally and intellectually engaging) • DATA COLLECTION: Knowledgeable and informative dialogue or conversation with 5-10 participants • DATA ANALYSIS: Open, tentative, intuitive and meaningful • COMMUNICATING FINDINGS: Biographical sketches

26. GROUNDED THEORY • PURPOSE: To increase understanding • PROCESS : Spend time interacting with participants • DATA COLLECTION: Field work • DATA ANALYSIS: Interpretational, structural, and reflective • COMMUNICATING FINDINGS: Descriptive and narrative

27. ETHNOGRAPHY • PURPOSE: To describe accurately the lived experiences of people – the relationship between culture and behavior. • PROCESS : Interactive • DATA COLLECTION: Participant observation, interviews and artifacts • DATA ANALYSIS: Comparing/contrasting • COMMUNICATING FINDINGS: Holistic description of people being observed

28. ENDOGENOUS • PURPOSE: To yield insider perspective through involvement of subject as researcher • PROCESS : Subjects of inquiry become the researchers – control of research plan is relinquished • DATA COLLECTION: Conducted by “insiders” of the culture • DATA ANALYSIS: Interpretational, open, and reflective • COMMUNICATING FINDINGS: Descriptive and narrative (from the subjects’ perspectives)

29. Mixed Methods • The research approach in which both quantitative and qualitative methods are used • Compatibility thesis • Position that quantitative and qualitative research methods and philosophies can be combined • Pragmatism • Philosophy focusing on what works as the criterion of what should be viewed as tentatively true and useful in research and practice • Questions to be answered when using a mixed design • Should you primarily use one methodology or treat them equally? • Should phases of study be conducted concurrently or sequentially? Christensen et al., 2014

30. KEY CONCEPTS • Fixed vs. Emergent Designs • Fixed: Use of Mixed Methods is Predetermined by Investigator • Emergent: Use of Mixed Methods arise as a result of issues that arise after research is underway

31. CORE DESIGNS • Convergent (Concurrent/Parallel) • Intent is to compare or combine results to obtain a more complete understanding of a problem • Explanatory Sequential • Qualitative results help to inform or explain quantitative results • Exploratory Sequential • Quantitative study designed based on qualitative results • Fixed: Use of Mixed Methods is Predetermined by Investigator • Emergent: Use of Mixed Methods arise as a result of issues that arise after research is underway

32. Three Core Mixed Methods Designs Creswell, 2018

33. Strengths Christensen et al., 2014

34. VALIDITY • Inside – outside validity • Present when the researcher provides both the insider and objective outsider perspectives • Weakness minimization validity • Present when the researcher compensates for the weakness of one approach through the use of an additional approach • Sequential validity • Making sure that the ordering of quantitative and qualitative components in a sequential design does not bias the results Christensen et al., 2014

35. VALIDITY • Sample integration validity • Researchers must not treat the quantitative and qualitative samples as equal, but, instead, draw appropriate conclusions from each sample • Multiple validities • Making sure your mixed methods study meets appropriate quantitative, qualitative, and mixed methods validity types Christensen et al., 2014

36. DESIGN SCHEME • Based on two dimensions: • Time Order • One of the two dimensions used in MM design matrix; its levels are concurrent and sequential • Paradigm Emphasis • One of the two dimensions used in MM design matrix; its levels are equal status and dominant status Christensen et al., 2014

37. TIME ORDER & PARADIGM EMPHASIS Morse, 1991

38. TYPES OF QUESTIONS BY DESIGN Depoy & Gitlin, 1998

39. Mixed Methods Approach:Example 1 QUANT Qual Evaluating Hope and Hardiness in Caribbean Immigrant Women Operationalize Variables: Scales Random selection of subgroup from top 10% of hardiness scores Analyze interviews by identifying common themes (e.g. frequency)

40. Mixed Methods Approach:Example 2 QUAL Quant Developing a Measure Begin with Focus Group Pull out Themes Develop Questions Factor Analysis Pilot Testing

41. Mixed Methods Approach:Example 3 QUAL + QUANT Small sample of middle school students in unique setting Ideal context to “hear” from members of marginalized group Culturally informed, strengths-based approach