1 / 28

COLLOQUIUM ‘Evaluating task-based language programs’ Evaluation of TBLT in Flanders

COLLOQUIUM ‘Evaluating task-based language programs’ Evaluation of TBLT in Flanders. Machteld Verhelst Lancaster, TBLT 2009. Content. Background: in what context? For whom and why? What? How? Findings and uses Benefits and pressure points. BACKGROUND. Flemish government

amable
Download Presentation

COLLOQUIUM ‘Evaluating task-based language programs’ Evaluation of TBLT in Flanders

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. COLLOQUIUM ‘Evaluating task-based language programs’Evaluation of TBLT in Flanders Machteld Verhelst Lancaster, TBLT 2009

  2. Content • Background: in what context? • For whom and why? • What? • How? • Findings and uses • Benefits and pressure points

  3. BACKGROUND • Flemish government > equal educational opportunities -> educational innovations Needs analysis (school language) -> learning goals (functional)= WHAT? • Improving language teaching: HOW? = TBLT (nationwide) -> materials development -> teacher training (preservice, inservice, coaching) & training of ‘meso-level’ (inspectors, counsellors, headmasters, teacher trainers) -> assessment

  4. IMPLEMENTATION PROGRAMS Innovations in primary schools with learners at risk Eg.: • Priority Policy Brussels (PPB) (2000-2009) • Brussels’ Border & Language Border Towns (2007-2010) • Circus Peekaboo (2002-2003) • L1 Ghent (2008-2012) • Europaschool (2009)

  5. For whom and why? Intended users: 1) field: policy makers -> meso-level -> teachers 2) educationalists: teacher trainers & material developers Evaluation Uses (Norris, 2009) • Understand teachers • Illuminate context • Improve programs • Encourage teacher agency • Ensure teaching success • Enable TBLT learning • Demonstrate outcomes Program implementation / TBLT implementation

  6. WHAT? • How is TBLT implemented? • Which implementation strategies work? • What do teachers think about (TB)LT? • What is the role of the whole school team/the meso-level (cooperation)? • What is the language policy in the school? • How powerful is the language learning environment? • What do teachers make of the tasks? • How do pupils react on the tasks? • What are the outcomes?

  7. How? General: ecologically valid: in authentic, naturalistic contexts ‘related to the situated realities of task-based teaching and learning’ (Norris, 2009) No effects of TBLT ‘pur sang’: always triangulating and contextualizing, reflecting the complex environments within which teaching and learning actually take place <> Flemish school effectiveness studies (large scale / written surveys -> decontextualized)

  8. How? Data: • Meso-micro • Multivariate • Qualitative and quantitative • Broader than tasks: ‘the powerful (language) learning environment’

  9. Informants at the meso – micro level: School - class - learner • Headmasters • counsellors • Teachers • Pupils • (parents) Control groups

  10. Multivariate • Input variables (background characteristics of both learners and teachers) • Process variables (teaching quality) • Output variables (language proficiency) Longitudinal: start – inbetween – end Multiple cohorts

  11. Qualitative and quantitative Qualitative (case studies / selection of teachers/classes, eg. in 12 schools K3 and G1): • Semi-structured interviews about perceptions, behaviour and self competency regarding TBLT and the language policy in their schools. • Stimulated recall interviews • (interviews with learners, depending on their age) • Teacher logs

  12. Observations • Needed: the development of a standard of good task-based teacher practice, which meant translating a theoretical framework of task-based language education into objectively observable parameters of teacher behaviour • Devlieger & Goossens, 2007, CUP • > coaching (observation and feedback, Verhelst & Van den Branden, 1999) • Later adapted as a research instrument, and according to specific research questions, eg. The role of L1 in TBLT

  13. A POWERFUL LANGUAGE LEARNING ENVIRONMENT Verhelst, 2006

  14. Extract of observation instrument

  15. Value and limitations of the instrument • Valid manner of grasping the quality of teaching practice • Able to compare teachers • Efficient to analyse (tendencies pop out) • Use as a coaching instrument (self-reflection) • Reliability: subjective -> multiple coders (2) -> video-recordings • Representative? • Frequency vs. Quality • Quantifiable? (cf. Van Gorp, forthcoming)

  16. Quantitative • Assessment (standardised, TB-tests measuring language proficiency; four skills) • Written (online) surveys: whole population - teachers/headmasters/counsellors = about perceptions, behaviour and self competency regarding TBLT and the language policy in their schools - learners: perceptions & attitudes towards teaching methods/language policy/self-competence/wellbeing & involvement

  17. Example of a timeline

  18. Communication of the results WRITTEN • Written report for policy makers • If requested: synthesis with general tendencies for meso-level/school teams • Article, website, monthly newsletter ORAL • Presentation at colloquia… (policy makers/meso) • Presentation to the school team • Individual feedback to the teacher Cyclic: beginning-end ! All reports have explicit recommendations & guidelines for future program action

  19. FINDINGS -> USES Overall findings concerning implementation • TBLT promising on paper, but practical & ideological concerns -> No full implementation of the intended program Teacher as an ‘eclecticus’ • Lack of transfer to other parts of the curriculum Uses • Coaching strategies • ‘language the whole day long’

  20. FINDINGS BPP (Van den Branden, 2006): Teacher level: • Stronger orientation towards functional language goals • More functional arrangement of the classroom • Higher quality of the teacher’s input ->Teaching quality improved in the whole learning environment • Little group work & oral production • Need for control (teacher as a leader, not a guide)

  21. Findings -> uses Learner level • Pupils’ higher level of involvement and motivation • pupils’ results improved • Declining influence of background characteristics (home language, socio-economical status) (these findings were confirmed in 2007) Uses -> Confirmation of the positive effects of TBLT -> coaching & material development (eg. ‘Totemtaal’, TB speaking assessment instrument)

  22. Program: Sustained implementation of TBLT with coaching, support Evaluation as inquiry, Norris (2009) Program: Training + coaching + agency Program: Theoretical inservice training Program: TB training + syllabus support • Methods: • Pre-post student learning outcomes, teacher surveys, classroom observations • Methods: • Teacher survey • Training observation • Methods: • Teacher logs, interviews, classroom observations • Methods: • Coaching obs, classroom obs, coach/ teacher interviews Findings: +incorporation of TBLT correlated with higher Dutch L2 proficiency outcomes +3 year gains in DSL higher in TBLT intensive adopting schools ?mixed incorporation of TBLT across schools, teachers • Findings: • Transmission model • Short term (3 hrs.) • “Try that with my students”…Post-coursal depression! Findings: +awareness of TBLT +student enthusiasm -teacher adoption -teacher control -task complexity -groupwork Findings: +conscious decisions +TBLT adaptation +self-evaluation ?teacher control -transfer -groupwork

  23. Circus Peekaboo (Verhelst & Verheyden, 2003) • Attitudes towards L1 changed (teachers/parents) • Interaction richer (at home - in class) • Children started talking for the first time, became self-confident • Motivation/wellbeing higher Uses: research results persuaded other schools to adopt this program

  24. Ongoing / start of program • Brussels’ Border & Language Border Towns (De Maeyer et al., 2009) Speaking climate/correctness/negative attitude L1 x Declining scores on oral production test in K3 Use: coaching -> attitudes & positive climate • L1-Ghent (Bultynck et al., 2008) Poor learning environment at the three levels Use: Extra intensive coaching to improve interaction and functional use of L1 through TBLT

  25. Benefits and pressure points Benefits: Research as reform • Meeting the requirements of sound evaluation (Norris, 2009) • users as participants (Norris, 2009) in the inquiry – dialogue – actions -> ownership -> effect on practice (Cf. Kiely, 2009) • What is easily implemented vs. what does not get implemented at all -> implication for further actions - Arguments pro TBLT -> convincing ‘new’ stakeholders -> development of new materials…

  26. Benefits and pressure points Pressure points • Policy makers prefer/have more confidence in quantitative research methods (‘reliability vs. validity’) • Time consuming (observations, massive amount of data (collecting-analysing)) -> case studies: generalisabilty?

  27. Thank you QUESTIONS? Machteld.verhelst@arts.kuleuven.be

More Related