Vocabulary Testing: Diagnose, Track Growth, & Intervene

1. Vocabulary Testing: Diagnose, Track Growth, & Intervene Kathleen T. Williams, PhD dr.kathleenwilliams@gmail.com

2. Snapshot of Speaker • Began professional life as a second grade teacher in 1968 • Involved in test and instructional material development since 1989 • Has been a speech pathologist, classroom teacher, school psychologist, graduate & undergraduate instructor • Has authored tests for vocabulary, reading, and mathematics • Has a BS in speech pathology, MEd in elementary education, MA in educational psychology, and PhD in school psychology

3. Overview • Importance of assessing vocabulary • Conorming tests • Understanding scores • Normative or derived scores • Statistical significance of score difference • Prevalence of score difference • Tracking growth • Generating hypotheses • Receptive > Expressive • Expressive > Receptive • Qualitative analysis for designing intervention strategies

4. What is “Vocabulary”? • Lexical Store • Mental dictionary of known meanings of words and idioms • Some may be inaccurate or incomplete • Recognition vocabulary • For listening and reading • Productive vocabulary • For speaking and writing • Not the number of words one can decode in print • It is the concepts one can understand at some level

5. Levels of Word Knowledge • Unknown • Totally unfamiliar with the word • Never hear it • Never saw it in print • Personal example: • carapace • “under the cold carapace of her adult disappointments” • a protective, decorative, or disguising shell

6. Levels of Word Knowledge • Acquainted • Some familiarity with the word • May have seen or heard the word • Do not clearly understand it’s meaning • May understand its meaning in only one context • Could not use it precisely in writing or speaking • Personal example: • hessian • “The scullery maid backed away, her head down, wiping her dirty hands on her hessian apron.” • burlap

7. Levels of Word Knowledge • Unknown • Totally unfamiliar with the word • Never hear it • Never saw it in print • Acquainted • Some familiarity with the word • May have seen or heard the word • Do not clearly understand it’s meaning • May understand its meaning in only one context • Could not use it precisely in writing or speaking • Well known • Can comprehend the word when reading • Can comfortably use the word when speaking or writing • Understand its multiple means and uses

8. The Importance of Vocabulary • Testing vocabulary is part of most preschool screening batteries and Reading First K to 3 programs • Without a basic vocabulary, the potential for developing a reading problem is great • Not all children come to school with the basic vocabulary knowledge required to understand the directions and instructions of the classroom

9. Limited Vocabulary – Lasting Consequences • Hart & Risley (1995) demonstrated that early differences noted in children entering school remain static throughout their education • Many children with an underlying language disability also demonstrate poorly developed oral vocabularies • Plays a vital role in the development of an individual’s academic and career opportunities

10. Vocabulary & Literacy Vocabulary knowledge is strongly related to overall reading comprehension Correlations range from .85 to .90 If a word is decoded and pronounced but the meaning is not recognized, comprehension will be impaired A limited vocabulary represents a limited understanding of concepts Well-developed vocabulary skills and wide background knowledge help individuals comprehend more difficult and complex material

11. The “Matthew” Effect The more words you know, the more you can read The more you read, the more words you will learn For children starting school with a limited vocabulary, “more” reading does not result in “more” vocabulary (Stanovich, 1986)

12. PPVT-4 & EVT-2 • Conormed measures of receptive & expressive vocabulary • Ages 2 ½ through 90+ • PPVT-4 (Peabody Picture Vocabulary Test, Fourth Edition, Dunn & Dunn, 2007) • Plate of four pictures • Examiner says a stimulus word • Examinee points to one picture • EVT-2 (Expressive Vocabulary Test, Second Edition, Williams, 2007) • Single picture • Examiner asks a question • Examinee says one word that is a label or a synonym • Two parallel forms to each test

13. Conorming • Standardized on the same normative sample • Same population • Same point in time • Robust and stable measures of the relationship • Correlation of EVT-2 to PPVT-4 is .82 • Shared variance is 67%

14. Understanding Scores Limitations of raw scores Normative or derived scores Interpreting standard scores, percentiles, normal curve equivalents, and stanines Limitations of Age or Grade Equivalents Monitor progress or track growth with GSVs What to do with extreme scores

15. Limitations of Raw Scores • Not comparable from one test to another • Tests have different numbers and types of items • PPVT-4: 228 receptive items • EVT-2: 190 expressive items • May have similar but unique growth curves and varying score distribution • Within a test, the same raw score does not tell the same story

16. Normative or Derived Scores • Reference a examinee’s performance to a single group of interest • For PPVT-4 & EVT-2, you have a choice: • The referenced group can be based on the individual’s age (2 ½ through 90), or • The reference group can be based on grade and time of school year (fall or spring) • Normative scores include standard scores, percentile ranks, normal curve equivalents, and stanines

17. When to Use Age-based Scores Test results will be used as part of an evaluation that considers age-based norms as best practice, such as pre-referral evaluation as required by IDEA An examinee is not of the typical age range for his or her same-grade peers due to retention or other factors Testing is being completed in an atypical educational setting, such as a summer school program The formal educational background of the examinee has been sporadic or uneven for some reason, such as illness or transience

18. Interpreting Standard Scores • Indicates the distance of a examinee’s raw score from average, taking into account the variability of scores among examinees of that same age or grade • Mean of 100 and a standard deviation of 15 • SS of 85 to 115 = 68 percent of the group • SS below 85 or above 115 = ~16 percent • SS below 70 or above 130 = ~2 to 3 percent • Equal interval scale of measurement • Can add, subtract, multiple, & divide • Must get the same standard score next year to demonstrate a year of growth

19. Interpreting Percentiles • Percentile rank indicates the percentage of examinees in the reference group who performed at or below a specific examinee’s score • Percentile of 86 means the examinee scored as high or higher than 86 percent of the normative sample • Percentiles are an ordinal or rank-order scale of measurement • Not an equal-interval scale of measurement • Can not add, subtract, multiply, or divide

20. Normal Curve Equivalents • Referred to as NCEs • Range from 1 to 99, with a mean of 50 and a standard deviation of 21.06 • Are based on percentiles but have been statistically converted to an equal-interval scale of measurement • Can be averaged • Useful for reporting data

21. Interpreting Stanines • Stanines is a contraction of “standard nines.” • Single digit scoring metric with a mean of 5 and a standard deviation of 2 • Not an equal-interval scale of measurement • They provide a “band” interpretation rather than a single-score cut-off. • Look for stanines that represent different sections of the normal distribution

23. Developmental Norms Grade equivalents (GE) or age equivalents (AE) place an examinee’s performance along a growth curve across grade or age A GE or an AE is the grade or age at which a particular raw score is the average score GEs are expressed in tenths of a grade AEs are expressed by years and months

24. Limitations of Grade or Age Equivalents • An GE or AE does not necessarily mean that the examinee’s vocabulary knowledge is qualitatively the same as that of the average person at that grade or age • Should not be interpreted as a functional equivalent (different raw scores can indicate different strengths and weaknesses) • An examinee aged 12 with an AE of 9:6 may tend to know a different set of words than the average 9-year-old • Are not an equal-interval scale of measurement • Place examinees along a development continuum, which may not increase at regular intervals • Should not be used for diagnostic or placement decisions

25. Understanding Growth Scale Values Growth scale values (GSVs) are included for both PPVT-4 & EVT-2 Developed so progress could be followed over a period of years on the same continuous scale Standard scores, percentiles, NCEs, and stanines place an examinee’s vocabulary knowledge with a reference group (based on all those of the same age or grade) The GSVs provide a measure of an examinee’s vocabulary knowledge in reference to the entire range of growth across all ages or grades As an individual’s vocabulary grows, the GSV will increase An equal-interval scale Can be added, subtracted, or averaged Can be compared over time for many purposes such as RTI or program evaluation Allows tracking of individual or group progress

26. Tracking Growth - Example

27. What to Do with Extreme Scores • Very low scores • No interpretation of a raw score of 0 on EVT-2 • It is lower than a score of 1, but how much can not be determined • No interpretation of a raw score of 3 or lower on PPVT-4 • Corresponds to a score an examinee would obtain through random responding • Perfect or near perfect scores • Should be interpreted with caution • Report SS of 160 but note limitation on record form • Examinee’s vocabulary may not have been measured fully by the test

28. Qualitative Analysis of Results • Calculate the absolute value of the difference between PPVT-4 and EVT-2 standard scores • Determine a level of significance of the difference (Table B.7 in EVT-2 Manual) • NS, .15, .10, .05, or .01 • Example: If significant at .05, 95% probability did not occur by chance • Find the percentage of the normative sample with the obtained difference value (Table B.8 in EVT-2 Manual)

29. Example 1 • Age 7 years 1 month • PPVT-4 SS of 78 • EVT-2 SS of 84 • 6-point difference, not significant • Could have occurred by chance • Prevalence of difference • Occurred in more than 25% of the sample • Both SSs reflect a below-average performance

30. Example 2 • Age 8 years 8 month • PPVT-4 SS of 82 • EVT-2 SS of 73 • 9-point difference, significant at the .10 level • 90% probability did not happen by chance • Prevalence of difference • Occurred in over 25% of the sample • Fairly common • Both SSs reflect a below-average performance

31. Example 3 • Age 6 years 6 month • PPVT-4 SS of 81 • EVT-2 SS of 101 • 20-point difference, significant at the .01 level • 99% probability did not happen by chance • Prevalence of difference • Occurred in only 5% of the sample • Fairly uncommon • Receptive score below average but expressive score average

32. Receptive > Expressive • Good guesser; may have some knowledge of the word and can rule out one or two of the options • Good lexical store, but a problem with word retrieval • Broad experiential background; knows many labels • Strong knowledge of the prevailing culture; knows the current label

33. Expressive > Receptive • Lack of automaticity of word recognition skills • Deep processing vs. superficial • Limited experiential background; knows only one label • Different cultural or “world knowledge” base; can say an acceptable word • History of hearing impairment; has learned a specific set of labels

34. “The greatest mistake we can make, I believe, is to treat comprehension as a simple process.” Theory, Assessment, and Intervention in Language Disorders: An Integrative Approach, by Elizabeth Carrow-Woolfolk, 1988

35. Qualitative Analysis • Home versus school vocabulary • Vocabulary by part of speech • Crossover vocabulary sampling between the EVT-2 and PPTV-4 • Three-tier model

36. Home Versus School • Home words • Words of high or moderately high frequency that could be acquired through common life experiences • Labels children hear in a home environment rich in language experiences • Frequently read to • “Lots of talk”

37. Home Versus School • School words • Not common to home environments • Included in instructional lessons about exotic or foreign places • Labels for less concrete objects (geometric shapes) • Learned by reading newspapers, magazines, or other informational materials

38. Home Versus School • Home • Household objects & food items • Common objects found outside the home • Basic numbers & colors • Common domesticated & wild animals • Actions (verbs) • Attributes (adjective & adverbs) • Body parts

39. Home Versus School • School • Less common domesticated & wild animals • Geometric shapes • Musical instruments • Names of workers or occupations • Recreational items not found in all homes • Items specific to urban or rural environments • Scientific instruments & specialized tools • Geographic references

40. Home Versus School • Division is arbitrary • Above average score • Probably has extensive knowledge from both environments • Below average score • Lack of English vocabulary from home environment • Not profiting from educational environment

41. Vocabulary by Grammatical Form • Part of speech • Usage of word within the test • Noun, verb, or attribute (adjective or adverb) • Most early items are nouns • Focus on labeling • Actions & attributes • Verbs, adjectives, adverbs • More abstract than nouns • Classified as “Home” since typically used in conversation

42. Crossover Sampling • Receptive & expressive common words • PPVT-4 Form A & EVT-2 Form B • 57 possible items • PPVT-4 Form B & EVT-2 Form A • 59 possible items • EVT-2 items are open ended • Labeling items: PPVT-4 item listed as common to EVT-2 item only if the PPVT-4 stimulus word was the most frequent correct EVT-2 response (90%) • Synonym items: PPVT-4 item listed as common to EVT-2 item only if the PPVT-4 stimulus word was at least 70% of the correct response to the EVT-2 item

43. Three-tier Model Low-frequency words; technical words Tier 3 High-frequency, high-utility Tier 2 Beck, I. L., McKeown, M. G., & Kucan, L. (2002). Bringing words to life: Robust vocabulary instruction. New York: The Guilford Press. Known, common words Tier 1

44. Building Academic Vocabulary • According to Farber (1999), many high school students develop reading skills through a sixth-grade level but “haven’t actually read enough to develop the vocabulary or general knowledge that more advanced reading requires” (p. 1) • Children must have vocabulary to learn to read and comprehend, but then they gain vocabulary by continued reading

45. Building Academic Vocabulary • Good readers, as described by Lyon (1997), “bring strong vocabularies and good syntactic and grammatical skills to the reading comprehension process…” (p.5) • Poor readers • Lack of a strategy to learn new vocabulary • May exhibit an overdependence on context • Will expend too much effort to read each word • May use only partial knowledge to determine meaning

46. Provide language-rich activities involving listening and speaking Read or tell stories Ask questions Ask for a different word Ask examinee to retell story Tell jokes or tall tales Why is it funny? Unusual? Odd? Retell changing a key word

47. Words with Multiple Meanings • Interactive Word Walls • Brainstorm words that meet criteria (self-selection) • Provide contexts for meanings • Make associations from own existing knowledge and experiences • Apply words to real situations • Get children to think more metcognitively about their own vocabulary and the depth of their understanding about particular words • Have I heard the word used this way? • Have I seen the word used this way in print? • Have I used the word in my writing?

48. Link Meaning to Prior Knowledge Build new words using prefixes If centennial means 100 years, what does bicentennial mean? Compound words What are the two words in the compound word? wiretapping, scatterbrain, underline What do they mean separately? What do they mean when used together?

49. Questions? dr.kathleenwilliams@gmail.com