1 / 36

Barbara Louwagie – Piet Desmet

Barbara Louwagie – Piet Desmet. An empirical study on the influence of feedback on learning performance for second language learners of French using a web-based CALL environment: E xperimental design & results of the preliminary study. Objectives.

omar
Download Presentation

Barbara Louwagie – Piet Desmet

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Barbara Louwagie – Piet Desmet An empirical study on the influence of feedback on learning performance for second language learners of French using a web-based CALL environment:Experimental design & results of the preliminary study Eurocall 2006 – Granada

  2. Objectives • The development of (automatic) feedback is a central point in CALL. =>The general purpose of our study: feedback = effective supplementary opportunity for learning? • Aim of the preliminary study: optimalization of the design • => This presentation is about the search for an adequate design Eurocall 2006 – Granada

  3. Structure • Definition of the object of study • Preliminary study • New design Eurocall 2006 – Granada

  4. Definition of feedback “In the context of teaching in general, feedback is information that is given to the learnerabout his or her performance of a learning task, usually with the objective of improving this performance.” (Ur 2003) Eurocall 2006 – Granada

  5. Feedback? feedback evaluation quantitative qualitative (corrective) summative formative Eurocall 2006 – Granada

  6. Corrective feedback in formative evaluation • Aim: to create learning opportunities • Consists of • identification of errors committed • explanation of errors (cognitive strategies) • suggestions to correct the errors (Janssens ea 2003) Eurocall 2006 – Granada

  7. Advantages FB in CALL • immediate (Heift 2001) • individualized (Kornum 1989) • objective (Kornum 1989) • … Eurocall 2006 – Granada

  8. Verify if the feedback provided in the learning environment used at our university is effective • Definition of ‘efficacy’ or ‘effectiveness’“The power to produce effects or intended results. This suggests a strong causal relationship between an intervention, such as the use of a particular item of technology in a learning situation and a discernible change in the learning process, the learning climate or the learning achievement.” (Felix 2005) Eurocall 2006 – Granada

  9. Preliminary study: structure • Aim • Design • Subjects • Procedure • Learning Environment (Idioma-tic) • Statistical analyses • Results • Evaluation of the preliminary study Eurocall 2006 – Granada

  10. Preliminary study: aim • To verify the effect of different kinds of feedback on the score obtained for the final test • To test the design of the study Eurocall 2006 – Granada

  11. Design • Feedback configurations • Components & themes • Other variables Eurocall 2006 – Granada

  12. 5 feedback configurations, schematical representation no feedback feedback closed activity closed activity half-open activity general & specific FB intermediary FB general, specific & error-specific FB intermediary, general & specific FB Eurocall 2006 – Granada

  13. 5 themes, 3 components: • Speech acts • Telephone conversation • Vocabulary • Business vocabulary • Quantitative vocabulary • Grammar • Conjunctions • Grammatical orthography • Different feedback conditions for each of these themes Eurocall 2006 – Granada

  14. Type of FB as ‘within-subjects’ factor Eurocall 2006 – Granada

  15. + other variables • Personal variables(sex, level of proficiency in French, learning style (Kolb-test), educational background, …) • Situational variables(subject of the exercises, number of tries, time spent on a certain exercise, …) Eurocall 2006 – Granada

  16. Subjects • n = 107 • Undergraduate students in applied economic science & commercial engineering • Learners of French as a second language • 5 groups (n ~ 20) • Stratification solely based on current branch of study, otherwise at random Eurocall 2006 – Granada

  17. Procedure Duration: 9 weeks Week 1 • (short demo of Idioma-tic) • Orientation test itemset 1, 50 MC-questions Week 3 • Practice session 1 (1 ½ hour) itemset A1, A2, A3, A4, A5 (± 30 questions) Week 5 • Practice session 2 (1 ½ hour) itemset B1, B2, B3, B4, B5 (± 30 questions) Week 7 • Practice session 3 (1 ½ hour) itemset C1, C2, C3, C4, C5 (± 30 questions) Week 9 • Evaluation test itemset 2 (5*(3+3) items) Eurocall 2006 – Granada

  18. Learning environment • Authoring tool IDIOMA-TIC • Tracking and logging system Eurocall 2006 – Granada

  19. Tracking & logging

  20. Overview of activities (± 2000)

  21. Statistical analyses* • Cross-tabulations (tabulate procedure) • Correlations (corr procedure) • Multi-level analysis (REML-algorithm**) * Many thanks to M. Vandewaetere ** residual restricted maximum likelihood Eurocall 2006 – Granada

  22. Results: Descriptive statistics Cross-tabulation (tabulate procedure)Distribution for sex, high school curriculum and current curriculum Eurocall 2006 – Granada

  23. Distribution for exercise series and current curriculum Eurocall 2006 – Granada

  24. Analytic statistics Correlations (corr procedure) • Indication of influence of FB, but just below statistical significance • Some obvious significant results: • Correlation between the number of exercises made and the final score ( = 0.13, α = 0.0036) • … Eurocall 2006 – Granada

  25. Multi-level analysis I Multiple themes per subject are measured (5). Themes are clustered for any single student. • Feedback has a significant effect (F(4,415) = 2.99, p < 0.05 ) • Theme has a significant effect (F(4,415) = 69.84, p < 0.0001) Eurocall 2006 – Granada

  26. Additional remarks • The foregoing analysis is not complete • There are repeated measuring moments (3) for any single student.The scores obtained during these measuring moments for a single subject will correlate more strongly than the scores of different subjects on a single measuring moment. • The following analysis takes into account the repeated mesures design. Eurocall 2006 – Granada

  27. Multi-level analysis II Multiple themes per subject are measured (5); moreover, each theme is measured three times. Themes are clustered for any single moment, and the moments are clustered for any single student. • There is a difference between the three measuring moments (F(2,202) = 68.55, p < 0.0001) • There is a difference between the types of feedback (F(4,424) = 1456.89, p < 0.0001) • There is a difference between the themes(F(4,424) = 114.39, p < 0.0001) Eurocall 2006 – Granada

  28. These significant main effects (previous slide) can, however, not be interpreted unequivocally, • because the two-way interaction terms are significant • feedback*moment, F(8,768) = 15.52, p < 0.0001 • theme*moment, F(8,720) = 9.17, p < 0.0001 • Id. for three-way interaction • thema*feedback*moment: F(48,720) = 8.97, p < 0.0001 • Nevertheless, first indication of influence of FB • => Optimize the design to obtain significant results Eurocall 2006 – Granada

  29. Evaluation of the preliminary study • Some practical problems • Over-parametrization (1000 cells per subject) => many interaction effects • Only one observation per cell: 2 themes can not be compared • Stratification solely on the basis of current curriculum • Personal variables • Maintain these variables • Level of proficiency in French: Orientation test: OK • Learning style test – Kolb test: not validated psychometrically speaking, the results can not be validated Eurocall 2006 – Granada

  30. Exercise series • Re-usable • Pay attention to differences in the level of difficulty between themes (grammatical orthography is comparatively easier than the four other themes) • Authoring tool: • Stable learning environment • Tracking & logging: no scoring provided for half-open assignments Eurocall 2006 – Granada

  31. New design • Add qualitative study (questionnaires) • Feedback configurations: maintain 5 types • Themes: simplification: 5 => 3 themes • Speech acts: telephone conversation • Vocabulary: business vocabulary • Grammar: conjunctions • Other variables: maintain and extend: • Self-evaluation as to knowledge of and attitude towards French (motivation) • Familiarity with the on-line tool • … Eurocall 2006 – Granada

  32. Type of FB as a ‘between-subject’ factor Eurocall 2006 – Granada

  33. Procedure • Orientation test itemset 1, MC questions • Exercise moment 1 itemset A1, A2, A3, A4, A5 • evaluation 1 itemset 2 (20 closed assignments, 20 half-open assignments) • Exercise moment 2 itemset B1, B2, B3, B4, B5 • evaluation 2 itemset 3 (different from 2) • Exercise moment 3 itemset C1, C2, C3, C4, C5 • evaluation 3 itemset 4 (different from 2 & 3) Eurocall 2006 – Granada

  34. Extended stratification: • current curriculum, • sex, • score for orientation test, • educational background, … • Each time trial run • Items at random, for 2 reasons: • Concentration effect • Time lapse Eurocall 2006 – Granada

  35. More information Presentation: http://www.kuleuven-kortrijk.be/ALT/ Piet Desmet Professor of French and Applied Linguistics K.U.Leuven (Campus Kortrijk) Piet.Desmet@kuleuven-kortrijk.be www.ling.arts.kuleuven.ac.be/franling_e/pdesmet Barbara Louwagie K.U.Leuven Campus Kortrijk Barbara.Louwagie@kuleuven-kortrijk.be Eurocall 2006 – Granada

More Related