David Lohman The University of Iowa facultycation.uiowa/dlohman

1. The Role of Nonverbal Ability Tests in Identifying Academically Gifted Students: An Aptitude Perspective David Lohman The University of Iowa http://faculty.education.uiowa.edu/dlohman

2. Overview • Background • Nonverbal tests • Advantages • Disadvantages • Understanding abilities • Aptitude perspective • For minority students • Recommendations

3. Background

4. Why use nonverbal tests? • Measure abilities in ways that are fair to all students • Increase the diversity in programs for academically gifted and talented • Actively assist children who have not had the advantages of wealth or who have not from birth been immersed in English

5. Other factors to consider • Get the right kids, not just the right number • Especially critical for minority students • Next generation of writers, scientists, mathematicians • Crafting policy for the identification and development of a diversity of academic talents

6. Nonverbal Tests • Present visual stimuli (objects, line drawings) and • Require a nonverbal response (assemble a puzzle, point, fill in a circle) • “Nonverbal” describes the test, not the cognitive processes used to solve items • Involvement of verbal processes • Explicit (UNIT Analogic reasoning subtest) • Implicit (Figural Reasoning tests)

7. UNIT Analogy

8. Advantages • Reduced oral/written language load • Verbal knowledge, verbally mediated strategies can be in any language • Reduced mean differences between monolingual and bilingual students

9. Disadvantages • Pictorial tests • Deciphering line drawings • Shorter directions are not necessarily better directions 3. Unforeseen linguistic confusions

10. UNIT Analogy

11. CogAT Figure Analogy J K L M N :

12. Disadvantages • Figural reasoning tests • Task specificity greater than for V or Q

13. Task Specificity 1 Test 1

14. Task Specificity Test 1 Test 2 + =

15. Task Specificity Figural Verbal

16. Disadvantages • Figural reasoning tests • Task specificity greater than for V or Q • Large practice effects • Largest Flynn effect

17. Example of Flynn Effect Gains in Wechsler-Binet IQ for the U.S. White population.Sources J. Horgan (1995) and D. Schildlovsky.

18. Disadvantages • Figural reasoning tests • Task specificity greater than for V or Q • Large practice effects • Largest Flynn effect • Appearance of measuring something innate

19. Fluid-Crystallized Continuum (1)

20. Fluid-Crystallized Continuum (2)

21. Disadvantages • Figural reasoning tests • Task specificity greater than for V or Q • Large practice effects • Largest Flynn effect • Appearance of measuring something innate • Appearance of being culture fair

22. Culture fair? • Intuitively plausible but long discredited idea • Anastasi & Urbina (1997) Psychological Testing (7th ed.) • “no test can be equally fair to all cultures” • “nonlanguage tests may be more culturally loaded than language tests”

23. Cronbach quote • Cronbach (1990) Essentials of Psychological Testing (5th ed). • “no behavioral evidence is culture free.” • “the term ‘culture fair’ makes a dubious claim”

24. Scarr quote • Scarr (1994) In Sternberg’s Encyclopedia of Intelligence • “intelligence and ability tests sample human cultural knowledge, acquired (through) development.” • “Although tests such as the Raven Matrices may seem fair… puzzle-like tasks turn out to have their own limitations.”

25. CogAT Figure Analogy J K L M N :

26. Disadvantages • Figural reasoning tests • Task specificity greater than for V or Q • Large practice effects • Largest Flynn effect • Appearance of measuring something innate • Appearance of being culture fair • Distal predictors of academic success

27. Example of r = .6 Mathematics Ach.

28. Example r = .6 Mathematics Ach.

29. Example r = .6 Mathematics Ach.

30. Example r = .6 Mathematics Ach.

31. Construct Representation Verbal Nonverbal g Quantitative

32. What predicts academic achievement? Academic Achievement Verbal Reasoning Nonverbal Reasoning Quantitative Reasoning

33. Selecting students on the basis of a test of nonverbal reasoning ability would: • admit many students who are unprepared for advanced instruction in mathematics or science or other content-rich domains. • exclude many students who either have already demonstrated high levels of accomplishment in these domainsOR whose high verbal or quantitative reasoning abilities make them much more likely to succeed in such programs.

34. 1 1 40 40 50 50 60 60 99 99 Verbal Verbal N + Quantitative Quantitative Nonverbal Nonverbal N - Figural reasoning as an inaptitude?

35. Students with an N+ profile do less well in school than students with an N- profile Gohm, Humphreys & Yao (1998) find high spatial students do poorly on a wide range of academic outcomes. High N, high Q = engineer profile Cannot look at Nonverbal test alone Figural reasoning as an inaptitude?

36. Extravagant Claims, Unlikely Promises • NNAT claims • culture fair • a “very good” predictor of school achievement • small and inconsequential difference in mean scores for White, Black, & Hispanic students • identify equal proportions of high-scoring White, Black, & Hispanic students

37. Predict achievement? • r (NNAT, Reading) = .56 • r (NNAT, Reading in Spanish) = .32 • r (NNAT, Math) = .6 • r (CogAT, Reading) = .80 • r (CogAT, Math) = .81

38. Small mean differences between ethnic groups? equal proportions of high scorers? • Exceedingly implausible • e.g. NAEP differences 1 SD • Matrix format much studied • Sample is small and unrepresentative • 5.6 % Urban school children • More high SES Hispanics & Blacks • Numbers do not add up • W-B and W-H differences inconsistent • means < 100 for all three groups • SD’s all greater than 15

39. Demographics: Urbanicity

40. Demographics: SES

41. Mean W-B, W-H differences W-B W-H • Naglieri & Ronning (2000) 4.2 2.8 • Naglieri & Ford (2003) 3.2 2.0

42. Aptitude Perspective • Aptitude is • the degree of readiness to perform well in a particular situation or fixed domain. • Examples • Ability to comprehend instructions • To use previously acquired knowledge and skill appropriately • To make good inferences and generalizations • To manage one’s emotions

43. Academic accomplishment 1 Learning Context 2 Learning Context 1 Person characteristics

44. Academic accomplishment 2 Academic Accomplishment/ Expertise Performance Assessments On-grade and above-grade achievement tests Teacher grades/ evaluations

45. Predicting Math Achievement in Grades 1-12 from CogAT 6

46. Predicting Math Achievement in Grades 1-12 from CogAT 6 .23 .50 .15 Multiple R = .80

47. Predicting Reading Comprehension/ Vocabulary in Grades 1-12 from CogAT 6

48. Predicting Reading Comprehension/ Vocabulary in Grades 1-12 from CogAT 6 .63 .15 .07 Multiple R = .80

49. Predicting Reading Comp/ Vocab for All Students(Hispanics)grades 1-6 .66 (.72) .14 (.12) .06 (.04) Multiple R = .81 (.80)

50. Predictors of Achievement • The regression equations that best predict achievement in Reading, Mathematics, Social Studies, & Science from CogAT Verbal, Quantitative, and Nonverbal reasoning are the same for White, Black, Hispanic, and Asian-American students • Other investigators find the same (Keith)