Size and Depth of Vocabulary: What Does the Research Show?

1. Size and Depth of Vocabulary: What Does the Research Show? Norbert Schmitt

2. History • Two year review of vocabulary studies pertinent to size vs. depth • Results reported at AAAL 2012 • Manuscript submitted to Language Learning • 3 reviewers gave plenty of feedback • Latest version: focus on conclusions

3. Size and Depth • Vocabulary has often been characterized in terms of size vs. depth of knowledge • The distinction is widespread, but one depth is not easy to pin down • One reviewer states that depth is “the wooliest, least definable, and least operationalisable construct in the entirety of cognitive science past or present”

4. Size and Depth • It is time to start thinking about this distinction more rigorously • The various conceptualizations and measurements of depth make it difficult to start from a theoretical framework • So start from an empirical perspective to inform the debate: • Review all studies that have a measurement of size and at least one measurement of depth

5. Vocabulary Size • Size = the number of lexical items ‘known’

6. Vocabulary Size • Size = the number of lexical items ‘known’ (to some criterion of mastery, i.e. depth)

7. Vocabulary Size • Size = the number of lexical items ‘known’ (to some criterion of mastery, i.e. depth) • Every size test is also a depth test in the sense that a certain criterion of mastery must be met

8. Vocabulary Depth • Depth / Quality = How well do you know those items? • What can you do with those items? • Very broad: can be conceptualized and operationalized in a variety of ways

9. Size and Depth • The relationship between size and depth depends on: • How both are conceptualized • How both are measured

10. Conceptualizing Depth • Receptive vs. Productive Knowledge • Usually connected with the 4 skills

11. Reading, Writing, Speaking, Listening • Depth could be seen as how well word can be employed in the four skills • Vocabulary size correlates with all four skills • But little research which shows how well individuallexical items are employed in the skills

12. What is Involved in Knowing a WordNation (2001) Formspoken receptive/productive written word parts Meaning form and meaning concept and referents associations Use grammatical functions collocations constraints on use (register, frequency …)

13. Size and Depth • Knowledge of all of the word knowledge aspects taken together can be conceptualized as a relatively comprehensive depth of knowledge • But each aspect can be known to various degrees of mastery

14. Degree of Knowledge Schmitt, 2010a: 38

15. Vocabulary Depth • So, improving knowledge of any individual word knowledge aspect can be considered as adding to depth • Not an all-or-nothing concept • Anything that improves mastery can be considered additional depth

16. Vocabulary Depth Depth = • Degree of mastery of the form-meaning link • Polysemous word meanings • Derivations (word family members) • Collocations • Other word knowledge aspects but were not found in research in conjunction with a size measure

17. Vocabulary Depth • Depth in this conceptualization concerns individual lexical items • Only a small number of items can ever be measured, so unclear how generalizable the depth measures can be

18. Conceptualizing Size and Depth Meara and Wolter, 2004: 89

19. Lexical Organization • Concerns lexicon as a whole rather than individual lexical items • Depth could be seen as any of the word knowledge connections between items • But how to measure it? • Word associations • Difficult to interpret • Idiosyncratic to individuals • Good measure of organization? • WAF main measure, but ?

20. Lexical Fluency • Daller, Milton, and Treffers-Daller (2007) see fluency as a separate dimension • Can see fluency as depth (i.e. depth does not have to be knowledge, but can be seen as employability (skills, automaticity)

21. Recognition and Recall of the Form-Meaning Link • Laufer and Paribakht (1998) • VLT (form recognition) • PVLT (form recall) • Recall-recognition r=.89 (EFL: Israeli high school) .72 (ESL: Canadian university) PVLT ÷ VLT ratio EFL% ESL% Combined 77 62 2,000 94 84 3,000 76 58 5,000 62 63 10,000 46 44

22. Depth of Form-Meaning Link • As vocabulary size increases (and frequency level decreases), the recall/recognition gap increases • Learners are more likely to have both form recognition and form recall mastery at the higher frequencies (i.e. smaller gap) • Less likely to have form recall mastery at the lower frequency levels (i.e. form recognition mastery only) • Form recall lags both form recognition and meaning recall

23. Knowledge of Written vs. Spoken Word Forms • van Zeeland (2013) used a meaning recall interview to measure the written and spoken vocabulary knowledge of advanced L2 learners • Results showed a stronger correlation between written and spoken word knowledge than found by Milton and Hopkins (2006) (r = .85 vs. .68). • The relationship between learners’ knowledge of written and spoken vocabulary furthermore remained constant as overall scores increased • These results suggest that knowledge of vocabulary in the two modes may be more closely related than suggested by checklist test results (Milton and colleagues)

24. Depth of Spoken-Written Mastery • Some evidence that very small vocabularies might be mainly known phonologically • Somewhat larger vocabulary sizes shift to being known predominately orthographically ? Advanced learners tend to have relatively balanced spoken/written vocabularies, while lower-level students are prone to the type of imbalanced vocabularies found by Milton and colleagues?

25. Knowledge of Derivatives • Correlations between size and derivation/suffix knowledge • Schmitt and Meara (1997) r • recall of suffix derivation .27 and .35 • Recognition of suffix derivation .37 and .41 • Kieffer and Lesaux (2008) • recall of derivation .53 and .46 • Kieffer and Lesaux (2012) • recall of derivation .50 -.57 • Noro (2002) • recall function of affix .42, .54, .69 • Mochizuki and Aizawa (2000) • recall function of affix .54 - .65

26. Knowledge of Derivatives • Milton (2009) reviews Mochizuki and Aizawa’s results and concludes: • A vocabulary size of 3,000-5,000 families is necessary for affixes to be mastered • But even at 5,000 families, some affixes may not be known well

27. Depth of Derivative Knowledge • Size is only modestly related to knowledge of affixes and derivatives (system learned before items?) • Milton suggests that a threshold might exist (3,000-5,000 families?)

28. Knowledge of Collocation • Gyllstad (2007) high-proficiency Swedish ESL • VLT • 2 collocation tests • COLLEX 5: a 3-option form recognition test • COLLMATCH 3: a yes/no collocation judgement • Size - collocation (r=.90) • 10,000 families  >90% on both collocation tests • 5,000 families ≈ 85% • 3,000 families ≈ 70% • With larger vocabulary sizes, it is possible to recognize collocations well

29. Knowledge of Collocation • Laufer and Waldman (2011) review the literature and conclude: • receptive knowledge is related to general vocabulary knowledge • productive knowledge of collocation lags behind knowledge of individual words • the problem with collocations is not recognition, but in using them properly, i.e. productive mastery

30. Lexical Fluency • Laufer and Nation (2001) Israeli university • VORST (computerized, timed, modified VLT) • There was an increase in lexical access speed at a vocabulary size of around 5,000 word families • The larger the vocabulary size, the faster the access speed 2,000 – speed (r = -.38) 3,000 – speed (r = -.40) 5,000 – speed (r = -.50) 10,000 – speed (r = -.67)

31. Depth of Automaticity • The larger the vocabulary size, the faster the access speed • The larger the vocabulary size (and the lower the frequency level) the stronger the relationship between size and fluency • Hint of a threshold: there is an increase in lexical access speed at a vocabulary size of around 5,000 word families

32. Lexical Organization • Henriksen (2008) Danish high school Grades 71013 • VLT – association recognition .72 n.s. n.s. • VLT – association recall .85 .69 .55 • The relationship between size and association is stronger at lower grades than more advanced ones • Since the students had increasing vocabulary sizes at all three grades, we can also interpret the results to show a stronger size-association relationship for smaller vocabulary sizes than larger ones

33. Lexical Organization • Greidanus, et al. (2004) • Advanced Dutch learners of French • Form recognition and form recall • New WAF (paradigmatic, syntagmatic, and analytic) • Form recognition – association (r= .70) • Form recall – association (r= .81)

34. Lexical Organization • Size – association correlations .18 .36 .39 .43 .49 .51 .55 .61 .62 .69 .69 .70 .70 .72 .78 .81 .81 .82 .85 .85 .86 .89 • Overall, fairly strong correlations

35. Lexical Organization • Size – association correlations .18 .36 .39 .43.49.51.55.61.62.69.69 .70 .70 .72 .78 .81 .81 .82.85.85 .86 .89 Association recall Association recognition (WAF)

36. Lexical Organization • Size – association correlations .18.36.39.43.49.51.55.61 .62.69.69 .70 .70.72 .78 .81.81 .82 .85.85.86.89 Lower vocabulary size Larger vocabulary size

37. Lexical Organization • 3 studies showed a trend for larger vocabulary sizes having stronger size – association correlations • 2 studies showed a trend for larger vocabulary sizes having weaker size – association correlations

38. Lexical Organization • 3 studies showed a trend for larger vocabulary sizes having stronger size – association correlations • 2 studies showed a trend for larger vocabulary sizes having weaker size – association correlations • Does larger size relate to better lexical organization? • Evidence seems mixed at this point (Problems with measuring organization?)

39. Tentative Conclusions • Does greater vocabulary size relate to greater depth of vocabulary knowledge? • Yes, generally • But how strongly depends on what ‘depth’ is

40. Tentative Conclusions • How one views the size-depth relationship should depend on one’s purpose of use • If one wishes to discuss the nature of vocabulary in general, particularly with practitioners, then the distinction is useful

41. Tentative Conclusions • If one’s purpose is to characterize vocabulary knowledge in more precise terms: • theorizing • designing and interpreting research • assessment • Depth is probably too vague a term to be useful • Need to state lexical aspect addressed and focus on that

42. Tentative Conclusions • Virtually all aspects of vocabulary knowledge seem interrelated • This makes it difficult to discuss any particular conceptualization of depth in isolation • This makes it difficult to conceptualize overall depth as anything but the combined interrelationships between word knowledge aspects

43. Tentative Conclusions • The most widely-used vocabulary tests are size tests, and they typically describe their results as the number of words ‘known’ • But they do not define what this actually entails • Test developers need to explicitly state what correct answers on their tests entail, and what degree of depth they represent

44. Tentative Conclusions • There can be no clear conceptual distinction between size and depth • Size by definition is the number of lexical items ‘known’ to some criterion level of mastery • But the criterion will always be some measure of depth, and so the two will always be confounded

45. Questions / Comments Comments to help me understand size/depth better? ? Size?Depth? www.norbertschmitt.co.uk