Quantitative Resilience Research across Cultures and Contexts

1. Quantitative Resilience Research across Cultures and Contexts Fons J. R. van de Vijver

2. Outline • 1. General introduction • Tertium comparationis • Approaches: Absolutism/relativism/universalism • Identity of meaning • 2. Common problems of cross-cultural studies (and their solutions) • 3. Establishing similarity of meaning: • 3a. Bias and equivalence: Taxonomies • 3b. Examples • 4. Acculturation • Concepts and Models / Assessment • 5. Test adaptations • Concepts / Example

3. General Introduction • Conceptual core of cross-cultural studies • Aim is to compare constructs or scores • Is resilience the same across the globe? • Is Country A more/less resilient than Country B? • Comparison always implies some shared quality (tertium comparationis):If a comparison visualizes an action, state, quality, object, or a person by means of a parallel which is drawn to a different entity, the two things which are being compared do not necessarily have to be identical. However, they must possess at least one quality in common. This common quality has traditionally been referred to as tertium comparationis (Source: http://en.wikipedia.org/wiki/Tertium_comparationis).

4. Views on the Relation between Resilience and Culture • 1. Absolutism (“etic”) • Resilience refers to a universal set of characteristics that individuals use to cope with and thrive despite adversity • 2. Relativism • Resilience refers to a concept (dealing with coping and thriving) that is universally applicable; however, its manifestations may differ across cultures • Example: Zimmerman & Brenner (2010, referring to Ungar, 2006) • The conceptual foundation of resiliency theory can be applicable across cultures; the extent to which resources and assets are applied by youth in their experiences of adversity, however, may not be consistent across all contexts. • 3. Relativism (“emic”) • Resilience refers to basic concept of coping and thriving; however, link between resilience and cultural context is so close that cross-cultural comparisons of resilience are futile and superficial

5. Choice between models is often made on an ideological basis However, more productive to see absolutism and relativism as extremes along a continuum Empirical studies possible of adequacy of these viewpoints Cross-cultural evidence is vital for determining which viewpoint holds for a particular measure/construct

6. Part 2 What are common problems in comparative studies? Central problem: Identity of meaning

7. Common methodological problems of cross-cultural research and their solutions

8. Problem 1 Cross cultural differences in scores cannot be interpreted due to rival hypotheses Particularly salient in two-culture studies that do not consider contextual factors Solution: Anticipate on rival hypotheses by including more cultures or measuring contextual factors

9. Problem 2 Cross-cultural similarities and differences are visually (and not statistically) tested A common example is the absence of a test of similarities of internal consistency coefficients Solution Explicit tests of cross-cultural similarities and differences; e.g., simple test of similarity of independent reliabilities available

10. Test of Independent Reliabilities

11. Problem 3 Samples show confounding differences Particularly salient in convenience sampling Solution: Adaptation of study design and assessment of confounding differences

12. Problem 4 Means of different cultural groups are compared without assessing the equivalence Particularly salient when studying new instruments or working with cultures in which instrument has not been used Solution: Assessment of structural and metric equivalence; assessment of structural equivalence/differential item functioning should be a routine part of analysis, similar to routine assessment of internal consistency

13. Problem 5 Cultural characteristics are attributed to all individuals of that culture (ecological fallacy) Particularly common in studies of individualism—collectivism Solution: Awareness of distinction between individual-and culture-level characteristics Assessment of relevant characteristics, such as individualism—collectivism, at individual level

14. Problem 6 No check on quality of translation/ adaptation Check is often not reported or procedure is poorly specified (e.g., translation back translation has been used, but results of procedure are not reported) Solution: Awareness that translation back translation is not always the best possible method; other approaches, such as committee approach, may be more suitable More detail in reports about translation/adaptation procedure

15. Problem 7 Lack of rationale for selecting cultures Convenience sampling of cultures is by far the most common procedure in cross-cultural psychology; most common comparison is between Japan and the US Solution: Explain why the culture was chosen

16. Problem 8 There is a verification bias in studies of common paradigms Particularly salient in studies of individualism –collectivism Solution: More critical appreciation of the boundaries of the construct, more focus on falsification

17. Problem 9 There is a focus on the statistical significance of cross-cultural differences In the first and two related problems: Implicit goal of cross-cultural psychology is not the establishment of cross-cultural differences Focus on significance detracts attention from effect sizes Solution: Balanced treatment of similarities and differences; differences easier to interpret against a backdrop of similarities More effect sizes should be reported, such as Cohen’s d and (partial) eta squares.

18. Problem 10 Results are generalized to large populations, often complete populations of countries, although no probability sampling has been employed to recruit participants Particularly salient in convenience sampling of participants (often student samples) Solution More attention in reports for sampling frame and for consequences on external validity

19. Part 3a Bias and equivalence: Definitions of concepts A framework

20. (a) Bias and Equivalence Does the test measure the same attributes for all cultural groups? Can scores be compared across ethnic groups?

21. Bias: Taxonomy What is internal bias? General: dissimilarity of psychological meaning across cultural groups Practical: when cross-cultural differences do not involve target construct measured by the test Theoretical: a cross-cultural comparison is biased when observed cross-cultural differences (in structure or level) cannot be fully interpreted in terms of the domain of interest

22. Taxonomy of Bias

23. Construct Bias Partial nonoverlap of behaviors defining construct González Castro & Murray (2010): Criteria for resilience are based on studies with U.S. youth and adults, and one important cross-cultural issue involves how these criteria, as Westernized aspects of resilience, may or may not relate to resilience that is manifest in underdeveloped and/or non-Western countries.

24. Definition of happiness in individualistic and collectivistic countries? Example: Uchida, Norasakkunkit and Kitayama (2004):

25. Types and Sources of Method Bias Method bias tends to have a global influence on cross-cultural score differences (e.g., increment due to social desirability)

26. Item Bias (also known as differential item functioning, DIF) Informal description Differences in psychological meaning of stimuli, due to anomalies at item level More formal definition: An item of a scale (e.g., measuring anxiety) is said to be biased if persons with the same trait anxiety, but coming from different cultures, are not equally likely to endorse the item.

27. Example of Biased Item

28. Types of (un)biased items

29. Analysis of Variance and Item Bias Item behavior examined per item We do not test for cultural differences, but we test whether scores are identical for persons from different groups with an equal proficiency Note: regression approach quite similar (illustrated later)

30. Taxonomy of Equivalence Refers to level of comparability Is related to bias: Highest level of equivalence obtained for bias-free measurement

31. Types of Equivalence Three types: 1. “Structural” or “functional equivalence” 2. “Metric equivalence” or “measurement unit equivalence” 3. “Scalar equivalence” or “full score equivalence”

32. (a) “Structural” or “Functional Equivalence” Measurement of the same traits Various statistical tools available, e.g., exploratory factor analysis (with target rotation) confirmatory factor analysis nomological networks (particularly relevant when items/questions are not identical across cultures) Qualitative equivalence can be firmly established

33. (b) “Metric Equivalence”, “Measurement Unit Equivalence” Difference in offset of scales of cultural groups, equal measurement units Individual differences have a different meaning withinand across cultures: no problems with offset in intra-cultural comparison, offset has to be added in cross-cultural comparison Statistical tool: structural equation modeling (confirmatory factor analysis)

34. (c) “Scalar Equivalence” or “Full Score Equivalence” Complete comparability of scores, both within and across cultures; seamless transfer of scores across cultures Frequently taken as the aim of cross-cultural research

35. Comparability and Equivalence Levels

36. Part 3b Establishing similarity of meaning How to determine equivalence? How to determine item bias?

37. Many statistical procedures available for testing structural equivalence • Common approach: • Apply dimensionality-reduction technique • Compare underlying dimensions across cultures • Similarity of underlying dimensions is criterion for similarity of meaning

38. Testing Structural Equivalence: Exploratory Factor Analysis

39. Two procedures explained • 1. Pairwise comparisons • Compare all cultures in a pairwise manner • 2. “One to all” comparison • Compare all cultures to a global, pooled solution

40. Characteristics of pairwise comparisons • Strong point: much detail, all pairs compared • Weak point: computationally cumbersome, difficult to integrate • Characteristics of pooled comparisons • Strong point: maintains overview, integration • Weak point: can conceal subgroups of countries

41. Example Pairwise Data set: WISC-III administered in Canada and Netherlands/Flanders

42. Sample

43. 12 Subtests Picture Completion Information Coding Similarities Picture Arrangement Arithmetic Block Design Vocabulary Object Assembly Comprehension Symbol Search Digit Span

44. Analysis Steps Determine number of factors in combined sample Carry out factor analyses per group Compare factors across groups Note: analysis of scaled scores

45. 1. Determining Number of Factors

46. 1. Determining Number of Factors Scree plot suggests the extraction of a single factor Literature: Debate about 3 or 4 factors Hierarchical model of correlated factors Here: 4 factors

47. 2. Factor Analyses per group: Oblimin-Rotated Solution

48. 2. Factor Analyses per group: Oblimin-Rotated Solution

49. 3. Compare Factors across Groups Rotate one solution to the other Target rotations to deal with rotational freedom in factor analysis Evaluation by means of Tucker’s phi (factor congruence coefficient): similarity of factors up to multiplying (positive) constant (correct for differences in eigenvalues across cultures)

50. 3. Compare Factors across Groups Formula (x and y are loadings after target rotation of one to the other):