Topic 9: Cross Culture Issues In Assessment And Measurement

1. Topic 9: Cross Culture Issues In Assessment And Measurement

2. Cross-Cultural Assessment • Cross-cultural assessment has become a sensitive issue due to specific concerns regarding the use of standardized tests across cultures. • Two main problems associated with cross-national testing are test translation and test adaptation. • Before selecting an assessment instrument for use in counseling or research, counselors and researchers are trained to verify that the test is appropriate for use with their population. • Investigation of validity, reliability, and appropriate norm groups to which the population is to be compared. • The instrument must be validly adapted, the test items must have conceptual and linguistic equivalence, and the test and the test items must be bias free (Fouad, 1993; Geisinger, 1994). • Issues of test adaptation, test equivalence and test bias must be considered in order to fully utilize the benefit of cross-cultural assessment.

3. Test Adaptation • Two basic methods for test adaptation have been identified: • Forward Translation • the original test in the source language is translated into the target language and then bilinguals are asked to compare the original version with the adapted version (Hambleton, 1993; 1994) • Back-translation • the test is translated into the target language and then it is re-translated back to the source language. This process can be repeated several times. Once the process is complete, the final back-translated version is compared to the original version (Hambleton, 1994). • Each of these adaptation processes has their strengths and limitations.

4. Advantages Of Adapting Existing Instrument • By adapting an instrument, the researcher is able to compare the already-existing data with newly acquired data, thus allowing for cross-cultural studies both on the national and international level. • Adaptations also can conserve time and expenses. • Test adaptation can lead to increased fairness in assessment by allowing individuals to be assessed in the language of their choice • Allows for greater generalizability • Allows for investigation of differences among a growing diverse population. • .

5. Disadvantages Of Adapting Existing Instrument • Risk of imposing conclusions based on concepts that exist in one culture but may not exist in the other. • There are no guarantees that the concept in the source culture exists in the target culture (Lonner & Berry, 1986). • Misleading conclusion • if certain constructs measured in the original version are not found in the target population, or if the construct is manifested in a different manner, the resulting scores can prove to be misleading (Hambleton, 1994).

6. Test Equivalence • According to Lonner ( 1985), there are four types of test equivalence: • Functional Equivalence • Functional equivalence refers to the role or function that behavior plays in different cultures. One cannot assume that behaviors play the same role or function across cultures; therefore, assumptions made about the function of behavior in a cultural group must be verified. • Conceptual Equivalence • Conceptual equivalence refers to the similarity in meaning attached to behavior or concepts. Certain behaviors and concepts may have different meanings across cultures. • Metric Equivalence • Metric equivalence refers to the psychometric properties and indicates that the scales measure the same constructs in different cultures. • Linguistic Equivalence • linguistic equivalence refers to the actual translation process. ,

7. Issues On Test Bias • Another issue that must be considered in cross-cultural assessment is test bias. • The test user must ascertain that the test and the test items do not systematically discriminate against one cultural group or another. • Test bias may occur when the contents of the test are more familiar to one group than to another or when the tests have differential predictive validity across groups. • Culture plays a significant role in cross-cultural assessment. When a test developed in one culture are used with another culture there is the potential for misinterpretation and stagnation unless cultural issues are considered.

8. Types Of Test Bias • Three types of bias: • Construct Bias • Method Bias • Item bias or differential item functioning

9. Types Of Test Bias • Construct Bias • occurs if the construct measured is not identical across cultural groups. • Item bias or differential item functioning • Refers to distortions at item level. Biased items have a different psychological meaning across cultures. Suppose that the subjects’ responses in one cultural group are partly determined by social desirability for one item of a self-report inventory. Then, a comparison of total test scores across cultures would be invalid when this item is included.

10. Types Of Test Bias • Method Bias - Three types of method bias can be envisaged. • Sample bias - Incomparability of samples on aspects other than the target variable, eg. Intergroup differences in motivation caused by sample incomparability. For instance, subjects who have been frequently exposed to psychological tests will show less motivation than subjects for whom the instrument has high novelty. • Instrument bias - refers to problems deriving from instrument characteristics. • Administration bias - bias arises from administration problems, such as communication problems between interviewers and interviewees can easily occur, especially, when they have different first languages and cultural backgrounds.

11. Type of Bias Strategies • Construct bias Decentering (i.e., simultaneously developing the • same instrument in several cultures) • Convergence approach (i.e., independent within- • culture development of instruments and subsequent • cross-cultural administration of all instruments) • Construct bias and/or • method bias • Use of informants with expertise in local culture • and language • Use samples of bilingual subjects • Use of local surveys (e.g., content analyses of free- • response questions) • Non-standard instrument administration (e.g., • “thinking aloud”) • Cross-cultural comparison of nomological • networks (e.g., convergent/discriminant validity • studies, monotrait–multimethod studies, connota- • tion of key phrases) • Method bias Extensive training of administrators (e.g., • increasing cultural sensitivity) • Detailed manual/protocol for administration, sco- • ring, and interpretation • Detailed instructions (e.g., with sufficient number • of examples and/or exercises) • Use of subject and context variables (e.g., educa- • tional background) • Use of collateral information (e.g., test-taking • behavior or test attitudes) • Assessment of response styles • Use of test–retest, training and/or intervention stu- • dies • Item bias Judgmental methods of item bias detection (e.g., • linguistic and psychological analysis) • Psychometric methods of item bias detection (e.g., • Differential Item Functioning analysis) • Error or distracter analysis • Documentation of “spare items” in the test manual • which are be equally good measures of the cons- • truct as actually used test items

13. Culture Fair Test

14. What Is Culture Fair Test? • A culture-fair test/culture-free tests is a test designed to be free of cultural bias, as far as possible, so that no one culture has an advantage over another. The test is designed to not be influenced by verbal ability, cultural climate, or educational level. • It designed to assess intelligence (or other attributes) without relying on knowledge specific to any individual cultural group. • History: • The first culture-fair test, called Army Examination Beta, was developed by the US military during World War II to screen soldiers of average intelligence who were illiterate or for whom English was a second language. • Beginning the post war period, culture-fair tests, which rely largely on nonverbal questions, have been used in public schools with Hispanic students and other non-native-English speakers whose lack of familiarity with both English language and American culture have made it impossible to assess their intelligence level using standard IQ tests.

15. What Is Culture Fair Test? • Examples of Culture-fair tests currently administered: • Learning Potential Assessment Device (DPAD) • Culture-Free Self-Esteem Inventories, • Cattell Culture Fair Series - intended to assess intelligence independent of cultural experience, verbal ability, or educational level. • Culture-fair testing is a timely issue given current debate over bias in intelligence and educational testing as it affects students who can speak and write English, but who are unfamiliar with white middle-class culture.

16. Purpose of a Culture Fair Test • The purpose of a culture-fair test is to eliminate any social or cultural advantages, or disadvantages, that a person may have due to their upbringing. • The test can be administered to anyone, from any nation, speaking any language. • A culture-fair test may help identify learning or emotional problems. • The duration of the test varies for the individual types of tests available, but the time is approximately between 12-18 minutes per section (a test usually has two to four sections). • A culture-fair test is often administered by employers in order to determine the best location for new employees in a large company. • The wide variety of culture-fair tests available allows the administrator to select which area is most vital, whether it be general intelligence, knowledge of a specific area, or emotional stability.

17. Description Of A Culture Fair Test (CFT) • A CFT is a non-verbal paper-pencil test that can be administered to a young person (four years old). • The patient only needs the ability to recognize shapes and figures and perceive their respective relationships. Some examples of tasks in the test may include: • completing series • classifying • solving matrices • evaluating conditions • The culture-fair test is also often referred to as a culture-free test or unbiased test. • There are many variations of the test including class, economic, and intelligence tests. The threading theme among the various tests is their design to be culturally unbiased.