Koen Van Gorp Centre for Language and Education, K.U.Leuven koen.vangorp@arts.kuleuven.be

1. How effective are content-based tasks in supporting both SLA and knowledge construction at the end of Flemish primary education? Koen Van Gorp Centre for Language and Education, K.U.Leuven koen.vangorp@arts.kuleuven.be

2. Centre for Equal Educational Opportunities Research funded by the K.U.Leuven. One of four studies of a larger scale research by the Centre for Equal Educational Opportunities (CEEO) (3 centres at 2 universities in Belgium): Centre for Language and Education (K.U.Leuven) Centre for Experiential Education (K.U.Leuven) Centre for Diversity and Learning (UGent)

3. This study: research questions How effective are second language learners of Dutch at the end of primary school in both knowledge construction and second language learning (i.c. the academic register)? Can we explain the learning of subject content and the academic register by the students? This presentation: results of the quantitative analysis brief look at qualitative analysis

4. Challenge for SL learners • Language is simultaneously a means of learning subject content as well as an academic discourse practice that has to be learned. • Both subject content and discourse become increasingly abstract and decontextualized in the course of primary education. • Being able to construct knowledge efficiently and becoming a proficient user of the academic register are requirements for successful participation in secondary education.

5. Taking up the challenge: The case DNA Quasi-experimental, pretest-posttest design A task-based lesson unit as intervention in the lessons ‘World orientation’ (mixture of social studies and science education) Measuring effects of intervention Pretest: 1 day before lesson unit Posttest: 1 day after lesson unit Delayed posttest: min. 4 weeks after posttest Control group (2 classes, 29 students) and experiment group (5 classes, 100 students)

6. Participants in this study 5 sixth grade classrooms More than 85% of SL learners 3 types of classroom composition: homogeneous, mixed, heterogeneous Average language proficiency: average weak 100 students 93% are SL learners 45% girls and 55% boys 67% low socio-economic status 57% repeaters; only 43% at age level

7. Data collection Beliefs and reported behaviour questionnaires, interviews Actions Observations: naturalistic (week 1) versus experiment (The case DNA – week 2) Output, learning outcomes Test DNA, language proficiency test Schools Teachers Pupils Parents

8. Lesson unit: The Case DNA A task-based lesson unit about DNA and DNA-investigation developed by the researcher. Clear lesson objectives, tasks, reading texts and worksheets for cooperative learning activities, background reading for teacher. Introductory, core and broadening activity: good for 125 - 175 minutes of teaching. Teachers were told to reach the stated objectives, but were free to adapt the lesson unit to their own teaching style: add or drop content, etc.

9. The Case DNA: example • Perform tasks like: Solving a crime by comparing DNA-profiles • Finding the answers to the following questions: • Where can you find DNA? • Where do your genes come from? • What do genes do? • What is DNA-testing used for?

10. Have the teachers taught differently? T_1: "I think I adapted the activities to my own teaching style." “I performed them as I prepared them at home.“ T_2: "No.. No. I don’t think so. “ T_3: “Uh no.“ T_4: "I like doing group work using jigsaw and I have some experience with it, but we should use it more often. I do it sometimes, but these DNA lessons are even stronger. “ "I really really was able to perform the activities as I wanted to.” T_5: "I try to work like that."

11. 1 task-based unit – 5 teaching practices In common: Explanation of core content by teacher; Using texts as source of information; Using group work. Personal didactic choices: Specific teaching activities: placemat, quiz; Supportive versus leading role of the teacher; Using all texts or only a few texts; Increasing motivation: using current events, inventing a new challenging police case.

12. Task-based? • Although all 5 teachers did some ‘retasking’ and sometimes ‘detasking’ (Samuda, 2005, 2007) • In general the task-based character of the lesson unit was preserved (Van den Branden, 2006): • functional: language as a means to reach a motivating goal • motivational: goal, content, didactic approach • relevant: abstract, complex content • challenging: with maximal potential for (language) learning • promoting task-based interaction: peer interaction, teacher-pupil interaction

13. Teachers’ reactions • Beforehand, most teachers expressed their doubts because of the difficult lesson content • T_1: “I was a bit afraid of what it was going to give, but I thought it went better than expected.” • T_2: “I am really pleased.” • T_3: “I think most of it went reasonably well.” • T_4: “It was great. I thought it was fun and very interesting.” • T_5: “I really thought it went well.”

14. Teachers about pupils’ reactions • T_1: “Children responded very positively to the lessons and I had not expected that.” • T_2: “Yesterday in the refectory huh! One pupil of the fourth grade asked ‘Miss, what is DNA?’. It must be fascinating, because otherwise they would not talk about it to each other.” • T_3: “Overall I thought it was better than expected.” • T_4: “Positive. I think the commitment was very high. I have felt it.” • T_5: “They behaved like always. I found them sometimes to be disinterested. My pupils quickly loose focus. Some were motivated, some not.”

15. And the pupils themselves? • Almost all of the interviewed pupils reacted very enthusiastic about the lesson unit. • S thought it fun and interesting: “Uh, you're a spy, you must find out about DNA research and stuff” • M thought it was the most interesting content of the whole school year next to sex education. • Most pupils told the researcher that they had learned a lot.

16. Test DNA: test design 33 test items in which language and knowledge are partly integrated: 12 true or false statements (+ indication of certainty) Ranking terms: cel, chromosome, DNA, gene. Cloze task (productive): filling in words (i.e. scientists, investigation, clues, crime, blood). Matching task: match words (i.e. unique, identical, expert, culprit, suspect) with their definitions. Recognition task (receptive): labelling pictures, i.e. cel, chromosome, DNA, gene, DNA-profile. Application task: analysing a crime scene, what does a CSI detective need for a DNA investigation?

17. DNA test: validity & reliability Content validity: as strong as possible one-to-one relationship with the lesson unit DNA knowledge items linked to the lesson objectives. Language items (vocabulary) based on corpus analysis (reading texts in lesson unit). Reliability Test-retest: Pearson’s r = .65** Internal consistency Pretest: α = 0.66 Internal consistency Posttest: α = 0.69 Internal consistency Delayed posttest: α = 0.70 ** Correlation is significant at the 0.01 level (2-tailed)

18. Mixed methods study “mixes or combines quantitative and qualitative research techniques, methods, approaches, concepts or language into a single study” (Johnson & Onwuegbuzie, 2004: 17) Sequential analyses: Quantitative analysis: focus on learning outcomes and explanatory variables Qualitative analysis: looking at teacher-student interactions

19. Results Test DNA in E-C classes

20. Results Test DNA for E-C groups

21. Data structure requires multilevel analysis CLASS 1 CLASS 2 CLASS 3 CLASS 4 CLASS 5 CHILD 1 CHILD 2 … Pre Post1 Post2 … …

22. Stepwise multilevel analyses (1) Comparing experiment and control group: lineair growth model (Model 1) After controlling for important background variables like gender, SES, language profiency, language background, the experiment group outperforms the control group significantly in learning gains between pretest and posttest: 5.74 vs 1.41 points on test DNA.

23. Stepwise multilevel analyses (2) Looking at the experiment group: multivariate respons model (Model 2) The experiment classes do not differ in effectiveness of the intervention after controlling for explanatory variables. Within-group variances are larger than between-group variances. 8 explanatory variables have significant effects on the different test moments. We find significant differential effects with respect to learning gains (between pretest and posttest) for 5 variables.

24. Effects of 8 explanatory variables over time

25. 5 differential learning gains effects Higher SES pupils gain 1.16 points more. Pupils who say that they speak only Turkish at home gain 1.89 points more. The higher the language proficiency of the pupil, how less the pupil gains between pretest and posttest: every 7 points above average on language test = 1 point less on test DNA. The higher the teacher estimates the competency of the pupil, the more the pupil gains: 1sd above average estimate = 1.38 points. The more number of turns the pupil has during the intervention, the more the pupil gains: every 28 turns above average = 1 point.

26. Conclusions at the classroom level • The experiment group outperforms the control group: the intervention was effective. • Learning took place in all experiment classrooms. All teachers created a powerful learning environment.The different ways the teachers adapted and performed the task based lesson unit did not lead to differential learning gains between classrooms. • There is an effect of a group composition variable (average language proficiency), but the effect is constant over time. It has no effect on learning gains.

27. Conclusions at the pupil level • In classrooms with more than 85% SL learners, SL learners can process rather abstract content (DNA) in an academic register (informative texts) and they can learn a lot in a relatively short period (145 – 215 minutes of teaching) from a task based lesson unit. • After controlling for explanatory variables there remains a lot of unexplained variance between pupils. • Some pupils learn more than other pupils. • Looking at the pupils that learn a lot.

28. From quantitative to qualitative analysis • Identification of pupils whose learning gains are one standard deviation better than predicted by the model (based on residual scores). • Looking at the personal interactions between teacher and pupils of some of these pupils.

29. Theoretical background • Importance of classroom talk or dialogue in sociocultural theory (Lantolf, 2000; Mercer & Littleton, 2007) • “it [classroom talk] is the most important educational tool for guiding the development of understanding and for jointly constructing knowledge.” (Hodgkinson & Mercer, 2008: xi) • Some important features of classroom talk: • Long conversations, Shared understanding • Scaffolding, Agency, Questions • Feedback, Uptake (learner, teacher)

30. The case of participation • We already know that quantity of participation in the classroom interaction plays a role. • Can we explain the learning gains of pupils by looking at the quality of their classroom participation? And more specifically, by looking at the quality of the teacher-pupil interaction? • Overall hypothesis: the quality of teacher-student spoken interactions help explain (at least partially) the learning gains.

31. Work in progress Qualitative descriptions of students who perform better than expected still going on: The case of M Tracing M’s learning about genes A model student? Fitting sociocultural theory?

32. About genes – Occurance in the data Frequency: 6 T-M sequences When: whole-class activities + group work What: short and extended interactions between M and teacher Contents: see overview academic participation Contents: see transcript

33. About genes - Overview From the beginning ‘genes’ catch the interest of M – M asks for clarification 3 times + 3 more occurances T-M: Each human has 99.9% similar genes T and P explain an idiom containing ‘genes’ to M T: “Can you buy genes?” M: “Yes” He gets clarification during group work: Extensive explanation by the teacher about relationship cell, chromosomes, DNA and genes M as overhearer Co-construction T-M about functions of genes Interestingly: M’s answers can largely be traced to the co-construction that went on between T and M.

34. About genes - Transcript T_390: What does a gene do? M_48: It gets nutrients from your euh bread. T_391: gets food, [T focus to the whole group] Do you already received that lesson of nutrients. That it enters in the blood stream. [T focuses on M] What more does it do / M_49: And it says how you look / /like./ / If you have brown or blue eyes. T_392: / / uhum. / / Voila. [T nods] M_50: xxx or your grandparents are. T_393: Uhum. Would it do also something else than just nutrients, so it says how you look and your hair and your eyes and all that, and you say that it gets the nutrients from food. Would that be the only thing? What would it do more? M searches in the text. + So what you look like, you just said. M_51: Cut toenailsT_394: How that nails grow. What else? M_52: in a complex way T_395: How can you say that briefly? How can you say that briefly, what you have marked? [T indicates highlighted text] + / / it ensures how that / / how your body, your body works. Yes, that nails grow, that my hair grows, that it gets nutrients, that is how your body works. These are the two answers you've found.

35. About genes – Evidence of learning Test DNA 3 statements aboutgenes Interview questions by researcher and M’s answers: Q1: What do genes do? “they take care of how you look like. Whether you have blue eyes or brown eyes.” “Genes, they, well, for example, they get nutrients from the bread that you eat.” When asked for other aspects? “Well, how your nails grow again, how your nose looks like.” Q2: what is the most important thing that you learned? “That that genes, that genes determine what you look like and how your body works. And that, that actually, what chromosomes is and what cells are.”

36. About genes - Analysis Analysing the contents of M’s interview answers we see a clear connection with M’s output (‘own construction’) in respons to the teacher (in blue) and we find evidence of learner uptake (from the teacher) (‘co-construction’) (in red). Specific hypothesis: Long personal conversations between T and M are important training areas for M’s output: putting learning into words. They provide opportunities for learner uptake in order to finetune or align the knowledge as well as the academic register to express or explain that knowledge (~ Swain’s Output Hypothesis)

37. About genes – link with student beliefs During the interview M expresses uncertainty about the relations between the core concepts. This uncertainty turns up in his learning results: Test DNA: he still is not able to circle genes on a DNA-helix (test item: circle some part of the DNA picture) Specific hypothesis: the lack of personal interaction between T and M influences M’s learning (i.c. depth of knowledge processing). Teacher has explained particular content both in whole-class discussion and in the group of M, but not directly to M. So M has not been able to check his understanding.

38. Linking beliefs, actions and outcomes The evidence that personalised interactions between T and M are important for M’s deep level learning link up with M’s claim that personal support of the teacher helps him more than a more collective kind of support (whole-class or groupwise). M’s outspoken personality, enthousiastic participation and sense of agency in the classroom seems to thrive on the personal attention and follow-up from the teacher. The teacher seems to be aware of this fact, but is not always able or willing to accommodate M’s needs (for all kinds of reasons, e.g. having more than one student in the classroom).

39. Conclusion M No real conclusions yet, only conjecture The overall hypothesis works for M! “the quality of teacher-student spoken interactions help explain (at least partially) the learning gains.” According to sociocultural theories about learning M’s behaviour and subsequent learning gains should not come as a surprise. Indeed, M can be seen as kind of a model student: setting his own learning goals, relating school content to his own social world, his sense of agency and active participation, his engament in dialogue with the teacher and other students.