Nadaclair : Cut L2Marking Time in Half (and maybe they’ll learn at the same time!)

1. Nadaclair.com: Cut L2Marking Time in Half (and maybe they’ll learn at the same time!) Terry Nadasdi, Stéfan Sinclair, Denis Cousineau

2. Preliminary remarks • My focus is error identification • This is an activity on which teachers spend countless hours and dollars • Is it helpful? • Probably • This study is indirectly interested in this • Who is quantitatively better and by how much?

3. Overview of talk a) discussion of previous “research” b) discussion of methodology (who are the humans, what is the computer?) c) presentation of quantitative results d) discussion of results e) conclusions about which is better (and what that means)

4. Advantages some studies found checkers helpful for SLA since they find a large number of errors (e.g. Burston, 2001) b) one study suggests that they are good because they make learners reflect on language (e.g. Durel, 2003)

5. Disadvantages Too many false positives No L1 interface - Not designed for L2 learners - Not pedagogical in orientation

6. Research problems Small databases b) Unrealistic representation of human correction. c) Criticisms have usually been based on invented sentences, not on empirical data.

7. Our study Who is better? Humans or …. Computers?

8. Our Data - 30 random texts submitted to BonPatron.com - 1000 - 1500 characters (roughly 250 words), - Anglophones, residing in Canada (outside of Quebec)

9. The humans - Two University Professors - Over 20 years experience marking FSL texts - PhD in French linguistics - PhD in French literature

10. Error Legend sp: Spelling (any word caught by spell check); only one error per word codedagdn: Determiner/Noun agreement error, e.g.: “tes ami” agan: Adjective/Noun agreement error, e.g.: “premier chose”; past participle agreement, e.g.: “la chose que j’ai remarqué agsv: Subject/Verb agreement error, e.g.: mw: Missing word, e.g.: “Je veux _ il mange”.wf: Wrong word form (e.g. infinitive instead of past participle, etc.); generally any time a French word form (found in the dictionary), was used in place of another word form, e.g. il a manger instead of il a mange

11. Error Legend (Con’t) la: Lexical (anglicism), e.g. “avoir un bon temps”wc: Word choice (e.g. avoir instead of être; wrong preposition)p: Punctuation (capitalization, missing space, missing comma, etc.)wo: Word order (syntax problems), e.g.: je veux la el: Elision, e.g.: “je aime”

12. Example of human correction Le*agdn premier*agan chose que nous avons remarqué*agan a été*wf le temps*la que tu te chouches*sp. Plutôt de*wc 11h 30, tu devrais te couches*wf à 10h. Ce changement est le plus importante*agan de tout. Si tu te chouches*sp à 10h, tu peux te léver*sp à une heure matinale. Le*agdn deuxième chose *mw nous avons remaqué*sp*agan a été*wf que tu prendsd*sp l’*p autobus pour arriver à l'école très vite. Nous pensons que tu devrais marcher*la à l'école. Puis tu auras marcher*wf pour*la un*pkilometre*sp, tu seras en bonne forme. Probablement que tu n'auras pas le temps *mw récontrer*sp tes amis avant le début des cours *pmais ce n'est pas très improtante*sp*agan …

13. The computer: BonPatron! Designed to address many of the criticisms previously raised in software reviews

14. Basic Stats Daily page views: 120,000 Daily users: 30, 000 Countries: 183 Who: school kids, university students, private industry, sword makers, roller-skaters, judo enthusiasts, self-mutilators, horse lovers, etc.

15. Let’s have a look at this BonPatron of yours: www.pro.bonpatron.com

16. 1) Il a reste Process-writing approach

17. Table 1 (human results)

18. Table 2 (computer1)

19. Table 3 (computer2)

20. Table4 (grammar subcategories)

21. Table 5: results excluding space before punctuation mark:

22. Discussion what does human correction really look like? b) do students learn from human correction?

23. 2) “Enfin, exhaustif dans son repérage … [la médiation humaine] … est pratiquement infaillible même s’il peut parfois être influencé par des facteurs psychologiques (être tolérant ou agacé, se lasser ou même s’ennuyer) ou physiologiques (se fatiguer)” (Cordier-Gauther & Dion, 2003). Perfect in it’s ability to identify errors, human correction is almost infallible, even it can occasionally be influenced by psychological factors (to be accepting, or bothered, lose interest or even become bored or physiological (become tired).

24. In the context of the student – teacher marking experience, certain humans are at best slightly better at error identification, but computers may well be better at error remediation.

25. Computer advantages a) Tireless (and always available) b) Impartial c) Real-time feedback d) Explicit corrective feedback (with an example)

26. Vast majority of research shows the positive effect of corrective feedback on L2 written production - Ferris (1995, 1997, 1999, 2003, 2006) - Chandler (2000, 2003) Bitchener (2006, 2007) - Bitchener, Young & Cameron (2003)

27. I am in grade 7 and My French has improved because of BonPatron, I used to get c-- or c--- even F now i get c+ and sometimes c, and I hope to get a B some day! BonPatron you've changed my Life ! Geneviève Mes notes de Français (je prends un cours d'histoire en immersion au niveau 10e année) étaient en train de baisser, mais avec l'aide de votre site mon prof a noté beaucoup d'amélioration. Merci! Hannah Jackson A colleague of mine, gave the BonPatron site as a tool for her advanced translation class. She says she has never received assignments for that class with so few agreement errors! Cheers and many thanks to the both of you... Dana Paramskas, French Professor, University of Guelph

28. Thank you so much BonPatron you have helped my mark go up at least 20 percent now I have a happy B+ My teacher was really smart to tell you this site! Sarah. I use your website all the time. It is so awesome. I actually learn a lot of great things and I like seeing that my writing is improving because I get less and less little yellow and red marks. Keep it up, it's really great! Sarah

29. Concluding remarks - Previous accounts of human marking have not been realistic - Our study has shown that even when compared with super human correction, BonPatron.com fares very well (conservatively, it catches more than 70% of errors) - It is likely that the feedback BonPatron provides is more beneficial for learning the rules of written French

30. Thank you!