1 / 18

Learner corpus analysis and error annotation

Learner corpus analysis and error annotation. Xiaofei Lu CALPER 2010 Summer Workshop July 13, 2010. Overview. Analyzing raw corpora Error annotation Issues in corpus annotation Granger (2003). Analyzing raw corpora. Concordancing software GOLD AntConc Other software CLAN.

wash
Download Presentation

Learner corpus analysis and error annotation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Learner corpus analysis and error annotation Xiaofei Lu CALPER 2010 Summer Workshop July 13, 2010

  2. Overview • Analyzing raw corpora • Error annotation • Issues in corpus annotation • Granger (2003)

  3. Analyzing raw corpora • Concordancing software • GOLD • AntConc • Other software • CLAN

  4. Issues in corpus annotation • Annotation scheme and format • Annotation procedure • Annotation quality

  5. Annotation scheme and format • What are the categories you are using? • Linguistically consensual • Overspecification vs. underspecification • Use short, meaningful codes for your categories • Annotation format considerations • Compatible with annotation scheme • Facilitates corpus query

  6. Annotation procedure and quality • Annotator training • Scheme and format • Problematic cases and disagreements • Computer-assisted manual annotation • Stanford annotation tool • UAM Corpus Tool and NoteTab • Inter-annotator agreement • Cohen’s Kappa • Online Kappa calculator

  7. Granger (2003) • Learner corpora • Error annotation • Error statistics and analysis • Integration of results into CALL • Conclusion

  8. Learner corpora • What is a learner corpus? • Difference from traditional data in SLA • Difference from native language data • Frequencies • Errors • From error annotation to error detection

  9. Computer-aided error annotation • Dagneaux, Denness and Granger (1998) • Manual correction of L2 French corpus • Elaboration of an error tagging system • Insertion of error tags and corrections • Retrieval of lists of error types and statistics • Concordance-based error analysis • Tagging system • Informative but manageable • Reusable, flexible, consistent

  10. Error tagging system • Dulay, Burt & Krashen (1982) • System based on linguistic categories (e.g., syntax) • Surface structure alternations (e.g., omission) • Granger’s (2003) three-dimensional taxonomy • Error domain • Error category • Word category

  11. Error tagging system (cont.) • Error domain and category • General level: grammatical, lexical, etc. • Domains subdivided into error categories • Table 1, page 468 • Word category • A POS tagset with 11 major and 54 sub-categories • Makes it possible to sort errors by POS categories

  12. Error tagging system (cont.) • Correct forms inserted next to erroneous forms • Facilitates interpretation of error annotations • Allows for automatic sorting on correct forms • Tag insertion using a menu-driven editor

  13. Error statistics and analysis • Error frequency by domain or (word) category • Highest ranked domains: grammar and form • Error trigrams • Concordancers for searching error codes • AntConc • WordSmith Tools

  14. Integrating results into CALL • Goal: a hypermedia CALL program • Using NLP and Communicative approaches to SLA • Traditional and NLP-enabled exercises • Automatic error diagnosis and feedback generation • Error statistics and analysis used to • Select linguistic areas to focus on • Adapt exercises as a function of attested error types • Adapt NLP tools for error diagnosis

  15. Integrating results into CALL (cont.) • Most error-prone linguistic areas • Tense and mood, agreement • Articles, complementation, prepositions • Adapting exercises • Exercises reflect type of error-prone context • Formal errors through dictation and exercises targeting specific difficulties • Attention to punctuation

  16. Integrating results into CALL (cont.) • Adapting NLP tools for error diagnosis • Spell checker and parser • Handles orthographic, grammatical, syntactic, and lexical errors • Not punctuation, semantic, and tense errors

  17. Granger (2003) summary • Effective 3-tier error annotation system • Limited number of categories per tier • Versatile automated data manipulation • Limitations of error-tagging • Element of subjectivity in annotation • Focuses on misuse • Usefulness of error-tagged learner corpus • Error statistics helps understand learner interlang • Helps adapt pedagogical materials and programs

  18. Activity • Using the Stanford annotation tool • Annotate a short text using your own scheme, or • Annotate a short learner text using Granger’s (2003) scheme • Query the annotated text using AntConc

More Related