Potentials and boundaries of formative assessment: Insights from New Zealand

1. Potentials and boundaries of formative assessment:Insights from New Zealand Yongqi GuVictoria University of WellingtonNew Zealand peter.gu@vuw.ac.nz

2. The big question • What isassessment for?

3. Assessment purpose: norm- vs. criterion-referenced • To rank or select among the students: Norm-referenced • To see who can achieve set goals or standards: Criterion-referenced

4. Assessment purposes: Summative vs formative

5. Purposes of classroom assessment • To obtain info on how effective teaching has been • To provide feedback to learners: • Progress? • Weaknesses? • To give learners an incentive to study • To clarify to both teacher and learners what learners need to achieve in terms of: • what to learn • amount of learning • level of performance • …

6. Language assessment in China • Language assessment research • Review of 9 key journals • All assessment articles (total=91) published between 2001 and 2005 • Language assessment practice

7. Journals reviewed • Foreign Language Teaching and Research • Modern Foreign Languages • Journal of Foreign Languages • Foreign Language World • Foreign Languages and Their Teaching • Journal of PLA University of Foreign Languages • Foreign Language Research • Foreign Language Education • Foreign Languages Research

8. Journals reviewed (Chinese) • 外语教学与研究 • 现代外语 • 外国语 • 外语界 • 外语与外语教学 • 解放军外国语学院学报 • 外语学刊 • 外语教学 • 外语研究

9. Lable Topics A Proficiency test B Testing oral English C Testing listening D Testing writing E Testing reading F Integrative testing G Computer assisted testing H Reform in language testing I Band 4 and Band 6 Language assessment articles published in China: Categories

10. J Test analysis K Test taking strategies L Test format M SLA and testing N Authenticity O Curriculum and testing P Social aspects of language testing Q Pragmatic aspects of testing R Testing trends S Washback T Formative assessment U Using tests for diagnostic purposes Language assessment articles published in China: Categories

11. Language testing articles published in China (2001-2005)

12. Research concentration (2001-2005)

13. Four articles on formative assessment

14. Using summative tests for diagnostic purposes

15. Summary • Chinese researchers have concentrated on summative assessment (assessing OF learning), not on formative assessment (assessing FOR learning). • No article touched on criterion-referenced assessment from 2001 to 2005. • Chinese teachers have not extensively explored formative assessment as a teaching or learning tool for the classroom

16. Formative assessment • Formative assessment refers to assessment that is specially intended to generate feedback on performance to improve and accelerate learning (Sadler, 1998)

17. Assessment that can be formative is assessment that • Is embedded in a pedagogy of which it is an essential part (eg constructivist). • Shares learning goals with students. • Involves students in self-assessment. • Provides feedback which leads to studentsrecognising the gap and closing it. • Is underpinned by confidence that every student can improve. • Involves reviewing and reflecting on assessment data. (Harlen, 1998)

18. Three conditions for formative feedback • Knowledge of standards (knowing what good performance is) • Comparing these standards (desired level of performance) to the student’s own current level of performance • Taking action to close the gap between current and desired performance Sadler (1989)

19. How does feebback work?Feedback in a self-regulated learning framework Butler and Winne (1995) Teacher sets task Goals/criteria/standards Processes internal to student Domain knowledge Strategy knowledge Motivational beliefs Student goals Tactics & strategies Internal learning outcomes Paths of internal feedback Self-regulatory processes (Cognition, motivation & behavior) Externally Observable outcomes External feedback (Teacher/peers)

20. Does formative feedback work?Empirical evidence Black and Wiliam (1998)summarized over 250 studies, and concluded that: • Improving formative assessment raises standards, dramatically. And FA helps low achievers more than other students. • Something should be done about it inside the classroom • Teachers can improve formative assessment

21. Factors that influence the effectiveness of FA • Providing effective feedback to students. • Student’s active involvement in their own learning. • Adjusting teaching to take account of the results of assessment. • Recognising the profound influence of assessment on students’ motivation and self-esteem. • Ensuring pupils reflect on their learning and understand how to improve.

22. Inhibiting factors • A tendency for teachers to assess quantity and presentation of work rather than quality of learning. • Greater attention given to marking and grading, much of it tending to lower self esteem of students, rather than providing advice for improvement. • A strong emphasis on comparing students with each other, which demoralizes the less successful learners.

23. The New Zealand Experience • Standards-based assessment (SBA) at the national level • Qualifications awarded by completing sets of unit standards (desired learning outcomes), not in terms of years of learning, and not by comparison with other students. • Teachers design assessment tasks which relate to the specific learning needs of their learners and the context in which they are studying

24. The New Zealand Qualifications Framework • Aim: a single, coherent framework for New Zealand • Unit standard: a set of specific learning outcomes and the criteria for their assessment • Assessment/performance level: achieved/not achieved • Particular set of unit standards are required for certificates and diplomas at each level.

25. The NZ qualifications system: 10 levels

26. Level 1: Description

27. Level 3: Description

28. NCEA Assessment • National Certificate of Educational Achievement (NCEA):the National Qualifications Framework in schools • NCEA=Levels 1-3 in NQF: senior secondary school •  8 achievement standards for each school subject • Each stardard has a credit value and sits at a specified level on the NQF. • Credits are accumulated towards a qulification

29. Stadands-based Assessment in NCEA • Four levels of performance: • Excellence • Merit • Credit • Not achieved • Half of the standards for each subject are assessed by classroom teachers, the other half externally assessed nationally

30. An example: ESOL unit standard 17363 • Read independently information texts, using ESOL • Level: 3 • Credit: 5

31. Summary of SBA in NZ SBA in NZ is criterion-referenced assessment. It achieves a much better formative purpose than a summative one. This is because • SBA makes desired standards of learning explicit to both teachers and learners • It enables and encourages rich and frequent feedback from both the teacher and the students themselves • It does not emphasise the grade, and gives students multiple chances to achieve a standard.

32. What’s good about Standards-based assessment? 1 • Clear and specific goals and standards for both learners and teachers • Students assessed in terms of what they can do, and not how they compare with peers • Slow learners are given multiple chances to achieve.

33. What’s good about Standards-based assessment? 2 • Rich formative tasks during or at the end of particular units of work • peer, and self assessment based on observation, questioning, and feedback • give detailed information about students' learning needs • suggest appropriate teaching strategies to meet student needs

34. Problems with standards-based assessment 1 • Assessment by unit standards greatly increased teacher workloads as they designed tasks and implemented them, kept records of student achievement, and participated in the necessary moderation activities. • Unit standards are assessed on an achieved/not achieved basis. There is no provision for rewarding superior performance and academically able students may not be challenged to achieve to the best of their ability.

35. Problems with standards-based assessment 2 • Despite moderaton procedures, ensuringconsistent and fair assessment on a national basis is a tall order. • SBA is good for low-stakes, formative purposes, but is used for high stakes purposes such as certification.

36. Insights for language assessment in China • At the system level, the main purpose of foreign language assessment in China is probably always going to remain norm-referenced, summative, and selective. • At the pedagogical level, setting clear and realistic goals and standards for Chinese teachers and learners of English is a natural step we have to start. However, a NZ-style standards-based assessment is both impractical and unnecessary.

37. Insights for language assessment in China • At the classroom level, I see an urgent need for teachers to explore formative assessment tools such as self- and peer- assessment, portfolios that provide rich feedback information. • At the learner level, formative assessment will not only improve learning results, but also encourage self-reflection, self monitoring, and self-regulation.

38. Insights for language assessment in China • At the research agenda level, it is not a question of whether, but a question of how formative assessment should be made use of systematically by Chinese teachers and learners to improve learning.

39. Purpose of assessment? • If a major purpose of assessment is to inform teaching and learning, why aren’t we exploring formative assessment for its potentials in the foreign language classroom?

40. Thank you!