1 / 52

Tracking L2 Lexical and Syntactic Development

Tracking L2 Lexical and Syntactic Development. Xiaofei Lu CALPER 2010 Summer Workshop July 14, 2010. Outline. Lexical & syntactic complexity: The what and why Syntactic complexity in EFL writing Lexical complexity in EFL speaking. 2. Lexical and Syntactic Complexity: The What and Why.

adila
Download Presentation

Tracking L2 Lexical and Syntactic Development

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Tracking L2 Lexical and Syntactic Development Xiaofei Lu CALPER 2010 Summer Workshop July 14, 2010

  2. Outline Lexical & syntactic complexity: The what and why Syntactic complexity in EFL writing Lexical complexity in EFL speaking 2

  3. Lexical and Syntactic Complexity: The What and Why

  4. What is lexical and syntactic complexity Lexical complexity A multidimensional feature of language use encompassing lexical density, sophistication and variation (Wolfe-Quintero et al. 1998; Read 2000) Does not focus on errors, a dimension in Read’s (2000) conceptualization of lexical richness Syntactic complexity The range of forms that surface in language production and the degree of sophistication of such forms (Ortega 2003) 4

  5. Why measure linguistic complexity? First language acquisition & psycholinguistics Studies of L1 developmental sequence Objective measures of L1 developmental level Ordering experimental stimuli by complexity Relationship of complexity in childhood to symptoms of Alzheimer’s disease (Kemper et al. 2001) 5

  6. Why measure linguistic complexity? Second language acquisition Objective L2 developmental indices Assessing cross-proficiency differences Assessing effect of pedagogical intervention Tracking L2 learners’ linguistic development over time Relationship between lexical/syntactic complexity and proficiency claimed in many test rating scales 6

  7. Syntactic Complexity in EFL Writing Lu, X. (forthcoming 2010). Automatic analysis of syntactic complexity in second language writing. International Journal of Corpus Linguistics, 15. Lu, X. (forthcoming 2010). A corpus-based evaluation of syntactic complexity measures as indices of college-level ESL writers’ language development. TESOL Quarterly, 44(4).

  8. Outline Measures of L2 syntactic complexity L2 syntactic complexity analyzer Syntactic complexity & EFL writing development Summary 8

  9. Measures of L2 syntactic complexity Measures reviewed in two research syntheses Wolfe-Quintero et al. (1998) Ortega (2003) Selection criterion At least one previous study showed at least weak correlation with or effect for proficiency Issues among previous studies Variation in measure selection and definition Variation in experiment design Inconsistent results reported on the same measures 9

  10. Measures of L2 syntactic complexity Length of production Mean length of clause (MLC) Mean length of sentence (MLS) Mean length of T-unit (MLT) Sentence complexity Mean number of clauses per sentence (C/S) 10

  11. Measures of L2 syntactic complexity Subordination Mean number of clauses per T-unit (C/T) Mean number of complex T-units per T-unit (CT/T) Mean number of dependent clauses per clause (DC/C) Mean number of dependent clauses per T-unit (DC/T) 11

  12. Measures of L2 syntactic complexity Coordination Mean number of coordinate phrases per clause (CP/C) Mean number of coordinate phrases per T-unit (CP/T) Mean number of T-units per sentence (T/S) Particular grammatical structures Mean number of complex nominals per clause (CN/C) Mean number of complex nominals per T-unit (CN/T) Mean number of verb phrases per T-unit (VP/T) 12

  13. L2 syntactic complexity analyzer Input: plain English text Step 1: Parsing using Stanford parser Step 2: Retrieving & counting occurrences of Words, sentences, clauses, dependent clauses T-units, complex T-units Coordinate phrases, complex nominals, verb phrases Step 3: Computing ratios for the 14 measures Output: 14 syntactic complexity indices 13

  14. How counting is done Word: all non-punctuation tokens Other units: Tregex (Levy & Andrew, 2006) Define the units linguistically Formulate Tregex patterns matching the unit definitions Query the parse trees with the Tregex patterns Retrieve/count (sub)trees matching each pattern 14

  15. Definition and pattern examples Clause: subject + finite verb (Polio 1997) ‘S|SINV|SQ < (VP <# MD|VBP|VBZ|VBD)’ Dependent clause: adverbial, adjectival or nominal clause ‘SBAR < (S|SINV|SQ < (VP <# MD|VBP|VBZ|VBD))’ 15

  16. Evaluation Experiment setup 40 essays from the Written English Corpus of Chinese Learners (Wen et al. 2005), average 315 words Written by English majors in four-year colleges in China 20 used for training, 20 for testing Two annotators counted unit occurrences in the essays Inter-annotator agreement Evaluated on 10 essays F-score for unit identification: .907 (CN) - 1.000 (S) Correlations of complexity ratios: .912 (CT/T) - 1.000 (MLS) 16

  17. Unit identification results on test data 17

  18. Correlations of complexity ratios 18

  19. Error analysis Attachment and conjunction scope errors e.g., benefit a lot from [the Internet in academic study] More reliable in identifying higher-level units: S, C, T, CT Learner errors not a major cause for problems Advanced EFL learners Idiomaticity vs. grammatical completeness Some errors do not lead to structural misanalysis 19

  20. Syntactic complexity & EFL writing development Research questions The WECCL corpus Results Summary 20

  21. Research questions Effect of sampling condition Measures discriminating proficiency levels Magnitudes for differences to be significant Relationships between measures Patterns of development for the measures 21

  22. The WECCL corpus Essay length: range=[89, 892], mean=315, sd=87 22

  23. Effect of sampling condition Institution: sig. inter-institution dif. for All metrics using all data 12 metrics using Y1-3 timed arg essays Genre: sig. dif. between arg vs. nar for All metrics using arg & nar essays All metrics using timed arg & nar essays 13 metrics using timed arg & nar essays from ND 23

  24. Sampling condition effect (cont) Timing: sig. dif. between un/timed arg for 13 measures using all arg essays 11 metrics using arg essays from ND Data for other research questions 422 timed argumentative essays from ND 24

  25. Measures discriminating levels 3 showed sig. dif between first 3 levels MLC, CN/C, and CN/T 4 showed sig. dif between first 2 levels MLS, MLT, CP/C, and CP/T 5 showed sig. dif. between non-adjacent levels C/S, C/T, CT/T, DC/C, and DC/T 2 showed no sig. between-level dif. T/S and VP/T 25

  26. Significant magnitudes 26

  27. Relationships between measures Strong relationship between measures of the same type or involving the same structure MLS and MLT show weak-moderate correlations with subordination measures MLC shows low-weak negative correlations with subordination measures Length measures show moderate-high correlations with CN measures and weak-moderate correlations with CP measures CN and CP measures weakly correlated with each other 27

  28. Developmental patterns Measures with sig. positive changes Linear increase Y1-4: MLC, CN/C Increase Y1-3 (Y4=Y3): CP/C Increase Y1-3 (Y4<Y3, insig.): MLS, MLT, CP/T, CN/T Measures with sig. negative changes Linear decrease Y1-4: C/S Nonlinear Y1<Y2>Y3>Y4: DC/C, DC/T 28

  29. Summary of findings Important to control for the effects of relevant learner-, task- and context-related factors Seven measures recommended for future use CN/C, MLC: discriminates 2+ adjacent levels, linear increases CN/T, MLS, MLT: 2 adjacent levels; positive sig changes CP/C, CP/T: nonadjacent levels, positive sig changes Developmental prediction: complexification at the phrasal level vs. the clausal level 29

  30. Summary of findings (cont.) Smaller magnitudes than reported previously Clause as a potentially more informative unit of analysis than T-unit 30

  31. Limitations and future research Incorporating more measures and flexible definitions of structures into the analyzer Other conceptualizations of proficiency level Effect of L1 on syntactic development Relationship between developmental measures of fluency, accuracy and complexity at different linguistic levels 31

  32. Lexical Complexity in EFL Speaking Lu, X. (under review). The relationship of lexical richness to the quality of ESL speakers’ oral narratives.

  33. Outline Research goals and motivation Measures of lexical complexity Methodology Results Conclusion 33

  34. Research goals and motivation Research goals Automate lexical complexity analysis using 25 measures Evaluate the relationship of these measures plus the D measure to the quality of EFL speakers’ oral narratives Motivation Lexical complexity an important construct in L2 teaching and research Relationship between lexical complexity and proficiency claimed in many test rating scales 34

  35. Measures of lexical complexity Lexical complexity measures proposed in language acquisition studies and reviewed in Wolfe-Quintero et al. (1998) Read (2000) Malvern et al. (2004) Measures of the following three dimensions Lexical density Lexical sophistication Lexical variation 35

  36. Lexical density Proportion of lexical words (Nlw/ N) (Ure 1971) Previous findings Lower in spoken than written texts (Halliday 1985) Affected by various sources (O’Loughlin 1995) Relation to L2 writing non-significant (Engber 1995) Inconsistent definition of lexical words All nouns and adjectives Adverbs with adjective base Full verbs (excluding modal/auxiliary verbs) 36

  37. Lexical sophistication Five measures examined LS1: Nslw / Nlw(Linnarud 1986; Hyltenstam 1988) LS2: Ts / T (Laufer 1984) VS1: Tsv / Nv(Harley & King 1989) CVS1: Tsv / sqrt(2Nv) (Wolfe-Quintero et al. 1998) VS2: Tsv2 / Nv(Chaudron & Parker 1990) 37

  38. Lexical sophistication (cont.) Previous findings LS1: NS-NNS dif sig (Linnarud 1986); non-sig (Hyltenstam 1988) LS2: sig pre-and post-essay dif (Laufer 1984) VS1: sig NS-NNS dif (Harley & King 1989) Varying definitions of sophistication 2000-word BNC frequency list (Leech et al. 2001) 38

  39. Lexical variation 20 measures examined 4 based on NDW NDW: Number of different words NDW-50: NDW in first 50 words of sample NDW-ER50: mean NDW of 10 random 50-word subsamples NDW-ES50: mean NDW of 10 random 50-word sequences 39

  40. Lexical variation (cont.) 7 based on TTR for total vocabulary Type token ratio (TTR) Mean TTR of all 50-word segments (MSTTR) LogTTR, Corrected TTR, Root TTR, Uber The D measure (McKee et al. 2000) 9 based on TTR for word classes T{LW, V, N, Adj, Adv, Mod} / Nlw Tv / Nv, Tv2 / Nv, Tv / sqrt(2Nv ) 40

  41. Lexical variation (cont.) Previous findings NDW and TTR useful, but affected by sample size Transformations of NDW & TTR not equally useful D claimed superior; results mixed (Jarvis 2002; Yu 2010) Mixed results for word class TTR measures No consensus on a single best measure 41

  42. Research questions How does LD relate to the quality of EFL speakers’ oral narratives? How do the LS measures compare with and relate to each other as indices of the quality of EFL speakers’ oral narratives? How do the LV measures compare with and relate to each other as indices of the quality of EFL speakers’ oral narratives? How do LD, LS and LV compare with and relate to each other as indices of the quality of EFL speakers’ oral narratives? 42

  43. Data Spoken English Corpus of Chinese Learners (Wen et al. 2005) Transcripts of TEM-4 Spoken Test data in 1996-2002 Task 2 data used: 3-minute oral narratives Students ranked within groups of 32-35 12 groups of data used (1999-2002; N=32-35 each) Only rankings available, but not actual scores Example topic (2001) Describe a teacher of yours whom you found unusual 43

  44. Computing the measures Preprocessing Part-of-speech tagging (Stanford tagger) Lemmatization (Morpha) Measure computation D measure: vocd utility in CLAN Type counting: w, sw, lw, slw, v, sv, n, adj, adv Token counting: w, lw, slw, v Computation of the other 25 ratios 44

  45. Analysis Spearman’s rho computed for each group X: test takes’ rankings within the group Y: Values of each of the 26 measures Meta-analysis to combine results from the 12 groups Students divided into 4 levels based on rankings Levels A, B, C and D ANOVA’s run to determine inter-level differences 45

  46. Analysis (cont.) Alpha level = .05 / 28 = .0018 Identification of discriminative measures Significant combined rho (p < .0018) Significant between-level differences with linear decreases from Level A to Level D 46

  47. Lexical density and sophistication 47

  48. Lexical density and sophistication (cont.) 48

  49. Lexical density and sophistication (cont.) 49

  50. Relationships among the dimensions Low to weak correlations among measures in different dimensions Lexical variation demonstrated strongest relationships to raters’ judgments of the quality of EFL speakers’ oral narratives 50

More Related