1 / 29

Three strategies for assessment in autonomous language learning

Three strategies for assessment in autonomous language learning. Joan Jamieson, Northern Arizona University, USA & Carol A. Chapelle, Iowa State University, USA. Three Strategies. Adaptivity Feedback Self-assessment. An Adaptive Strategy.

asta
Download Presentation

Three strategies for assessment in autonomous language learning

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Three strategies for assessment in autonomous language learning Joan Jamieson, Northern Arizona University, USA & Carol A. Chapelle, Iowa State University, USA

  2. Three Strategies • Adaptivity • Feedback • Self-assessment

  3. An Adaptive Strategy • Learner would benefit from more than one form of material • Computer should select appropriate form based on responses to questions

  4. Overview of LEA Beginning Reading Beginning Listening Beginning Writing Results & Recommendations Interest and Ability Finder Intermediate Reading Intermediate Listening Intermediate Writing Results & Recommendations Advanced Reading Advanced Listening Advanced Writing Results & Recommendations

  5. The Interest Survey • Select test form • Select recommendations

  6. Items on Interest Survey

  7. Example Strategies

  8. A Feedback Strategy • Learner benefits from total scores • Learner might benefit more from part scores

  9. Example Computing Total Score

  10. Part Scores Reflect Subskills • Tests are often made up of subskills • Each item can be coded according to subskill • Scores for subskills can be computed by including codes

  11. Table of Specifications

  12. Tags for LEO Tests TAG What the TAG means L listening LIN listening for information LID listening for ideas G grammar G1 grammar point 1 G2 grammar point 2 G3 grammar point 3 S speaking V vocabulary R reading P pronunciation P1 pronunciation point 1 P2 pronunciation point 2

  13. Tags in Script for Grammar Section

  14. Using Tags with System Variables • “score” yields percentage correct • score (tag) yields percentage correct for any items with a given “tag” • score (G2) yields percentage correct of 2nd point of grammar—expressions for suggesting

  15. Combining Tags and System Variables score (L | G | V | S | P | R) n/m1= “rawscore(LIN) / tqw(LIN)” n/m2= “rawscore(LID) / tqw(LID)” n/m3= “rawscore(G1) / tqw(G1)” n/m4= “rawscore(G2) / tqw(G2)” n/m5= “rawscore(G3) / tqw(G3)”

  16. Mock-up of Progress Report Screen Progress Report LEO 3 Test Learner’s name: Score: score (L | G | V | S | P | R) Language area Number correct/number of items Listening for information n/m1 Listening for ideas n/m2 Grammar (point1*) n/m3 Grammar (point2*) n/m4 Grammar (point3*) n/m5

  17. Screen Shot of Progress Report

  18. Using Tags to Report Scores

  19. A Self-Assessment Strategy • Learner may benefit by comparing his/her perspective of performance with score • Computer can collect self-confidence data along with performance data

  20. Example of Self-Confidence Item Was your answer correct? How sure are you? Click a circle below. Completely Not sure at all sure

  21. Superimposed Self-Assessment Item Was your answer correct? How sure are you? Click in a circle for each answer. 1. 2. 3. Completely Not sure sure at all

  22. Computing Average Confidence (Tarone and Yule, 1989) Circle clicked 5 4 3 2 1 total average confidence correct answers 20 5 3 2 0 29 4.52 incorrect answers 0 0 4 5 2 11 2.00 (20*5)+(5*4)+(3*3)+(2*2)+(0*1)/29 = 4.52 (4*3)+(5*2)+(2*0)/11 = 2.00 Tarone, E., & Yule, G. (1989). Focus on the language learner. Oxford, UK: Oxford University Press.

  23. Computing Self-Monitoring Index • Derived by subtracting self-confidence rating on incorrect items from self-confidence rating on correct items: 4.52 – 2.00 = 2.52 • Index ranges in value from 4 to - 4 • Messages could be provided instead of numbers

  24. Self-Assessment Superimposed onto Progress Report Self-Assessment: You seem to be aware of your own ability. When you gave the correct answer, you were very sure you were correct. When you gave the wrong answer, you were not too sure you were correct.

  25. Implementing Self-Assessment • Tag self-assessment items <SA> • Save value of “rawscore (SA)” separately for correct and incorrect items: • IF ANSWER = 1 THEN SAOK = SAOK + rawscore (SA) • IF ANSWER = 0 THEN SANO = SANO + rawscore (SA)

  26. Calculating Average Scores • AVGSAOK = SAOK / # CORRECT ITEMS • AVGSANO = SANO / # INCORRECT ITEMS • MONITORING INDEX = AVGSAOK-AVGSANO

  27. Example of Computing Self-Assessment Scores

  28. Three Strategies for Individualizing Assessment • Adapting level, content, and recommendations based on learner’s responses • Additional feedback in the form of diagnostic scores • Self-assessment to heighten learner’s metacognitive awareness

  29. Three strategies for assessment in autonomous language learning Joan Jamieson, Northern Arizona University, USA & Carol A. Chapelle, Iowa State University, USA

More Related