Enhancing Effective English Language Assessment in Secondary Schools Teacher Conference, Hong Kong, April 22 nd , 2006

1. Enhancing Effective English Language Assessment in Secondary Schools Teacher Conference, Hong Kong, April 22nd, 2006 Towards more effective assessment in Hong Kong secondary schools: Findings from the chalkface Chris Davison Faculty of Education, University of Hong Kong

2. From a testing to an assessment culture “Assessment is the practice of collecting evidence of student learning in terms of knowledge, skills, values and attitudes through observation of student behaviour when carrying out tasks, tests, examinations, etc”. (Curriculum Development Council, 2000)

3. From a testing to an assessment culture • Assessment practices in Hong Kong traditionally aimed to select students for education or employment (Biggs, 1995), but major educational reforms underway, with school-based and standards-referenced assessment being introduced as way to improve learning and teaching (Education Commission, 2000; CDC, 2000; HKEAA 2003). • Official adoption of UK Assessment Reform Group’s distinction between assessment for learning vs assessment of learning

4. From a testing to an assessment culture Concerted move to adopt school-based assessment (Stiggins & Conklin 1992; Black & Wiliam 1998; Brookhart 2003): • Integral part of teaching and learning • Teacher-mediated, evolving • Multi-modal (observation, inquiry, analysis, test, etc) • Co-constructed and dialogic • Context-dependent • Oriented towards individual improvement rather than system-wide comparisons

5. Integral part of teaching and learning

6. Teacher-mediated and evolving

7. Multiple and varied formats

8. Multiple and varied formats

9. Co-constructed and dialogic

10. Context-dependent

11. Individually-oriented

12. From a testing to an assessment culture http://cd.emb.gov.hk/basicguide/BEGuideeng0821/chapter05.html “Based on the beliefs that every student is unique and possesses the ability to learn, and that we should develop their multiple intelligences and potentials … there should be a change in assessment practices and schools should put more emphasis on 'Assessment for Learning' as an integral part of the learning, teaching and assessment cycle …

13. From a testing to an assessment culture … In other words, teachers should use assessments (e.g. as simple as effective verbal questioning, observation of student behaviour) and provide immediate feedback to enhance student learning in everyday classroom lessons. The focus is on why they do not learn well and how to help them to improve rather than just to use assessments to find out what knowledge students have learned”

14. From a testing to an assessment culture • Supported by the development of standards-referenced assessment frameworks and "authentic" alternative school-based assessment activities/tasks embedded within the instructional program, as well as more formal summative school-based assessment (eg. HKCEE 2007). • Such assessment seen as complementing the recent adoption of a more outcomes-oriented English syllabus.

15. From a testing to an assessment culture BUT … experience elsewhere (and in Hong Kong with TOC and TAS) demonstrate conclusively that the shift from a "culture of testing" to a "culture of assessment" (Gipps, 1994) is fraught with difficulties - theoretical , socio-cultural and, above all, practical - thus must be carefully monitored, evaluated and supported.

16. Our research (studies), 2004-? Aims: • To stimulate, support, document and evaluate the development of effective formative and summative assessment practices in English language teaching, with a particular focus on the HKCE 2005-07. • To identify and describe factors which may facilitate and/or hinder the connection of formative (and summative) assessment and feedback with learning and teaching

17. Our research (studies), 2004-? • 13 interlinked projects, part of a coherent and systematic programme of research and evaluation and professional development, funded by HKEAA, QEF, UGC and the Faculty of Education, HKU, http://web.hku.hk/~sbapro • Data collected from all Form 4 English teachers and students in all HK secondary schools through questionnaires, individual and group interviews, focus groups, classroom observations, video-recordings, self-reflections and retrospective analysis.

18. The research (studies), 2004-? • Research school-based, co-participatory and action-oriented: “Action research [rejects] the concept of a two-stage process in which research is carried out first by researchers and then in a separate second stage the knowledge generated from the research is applied by practitioners. Instead, the two processes of research and action are integrated” (Somekh, 1995) • Results consistently demonstrate that there needs to be more attention to developing teacher assessment (un)readiness, and to system-level support.

19. The contribution of theory: The stages of concerns model • Hall, George, and Rutherford’s (1977) Stages of Concern Model is one of best known individual-oriented models found in the educational literature which can be used to track the concerns of teachers adopting or implementing edcational innovations. • Hall et al. define concern as “the composite representation of the feelings, preoccupation, thought, and consideration given to a particular issue or tasks” (p.5).

20. The contribution of theory: The stages of concerns model Two key assumptions are made in the model: • Assumption 1: Teacher concern is a multi-dimensional construct. 7 distinct stages of concern assumed to exist irrespective of the nature of the educational innovation … Self-concerns: Awareness, Informational, Personal. Task concerns: Management. Impact concerns: Consequence, Collaboration, Refocusing.

21. The contribution of theory: The stages of concerns model

22. The contribution of theory: The stages of concerns model

23. The contribution of theory: The stages of concerns model

24. The contribution of theory: The stages of concerns model • Assumption 2: Teacher concern is a developmental construct. Initially, a teacher’s self-concerns are expected to be the most intense. As the teacher becomes more comfortable, task concerns begin to dominate. Finally, impact concerns become most important and the teacher starts to demonstrate a strong sense of professionalism and control. A teacher can move back and forth across several stages as they develop.

25. The contribution of theory: The stages of concerns model Analyzing teachers’ stages of concern and their paths of development helps us to evaluate teachers’ “assessment readiness” and the nature of support they require. However, teachers are not unfettered free agents, but strongly influenced by their own personal histories and beliefs (including perceptions of self-efficacy), and the implicit and explicit discourses and practices at the institutional and societal level: “All action is socioculturally mediated, both in its production and in its interpretation” (Ahearn, 2001, p. 112).

26. TEACHER KNOWLEDGE, BELIEFS AND ATTITUDESeg. Role of education; nature of assessment, teaching and learning; nature of (English) language and language learning, prior experiences of assessment/change, etc EXTERNAL PRESSURES eg. Syllabus and exam requirements, other educational reforms, parental/community expectations, etc SITUATIONAL FACTORS eg. Time, trust, collegial support, student numbers/characteristics, perceived impact of assessment, etc Teachers’ assessment orientation and decision-making processes ASSESSMENT PRACTICES (Davison, 2004, in progress, adapted from McMillan 2003)

27. Findings • Although a range of assessment activities are being practiced in the schools, they are mostly summative assessments for grading purposes. • Assessment activities like dictations, formal examinations are more common than portfolios, student-teacher conferencing and/or student-student conferencing, and self and/or peer assessment. • Alternative assessment activities are usually tried out on a more informal basis and account for a small percentage of marks.

28. Findings • In terms of assessment practice, teachers are still more likely to assess discrete skills (e.g. speaking, writing, reading, listening) rather than integrated skills (e.g. writing notes before speaking/oral presentation/ discussion). • Students tend to have more experiences of individually-based assessment rather than pair/group, self and peer assessment. • The focus of assessment is upon the final product rather than the learning process.

29. Findings • There is a wide range of teachers’ concerns about school-based assessment in both its formal and informal modes. • Our data suggest about 20-30% of teachers are still exhibiting Stage 3-type concerns about “alternative” assessments, about another 20-30% are at Stage 5, or higher, with the rest still demonstrating characteristics associated with Stage 4, but with clear shift in the upward direction, especially after the experience of active group experimentation and sharing.

30. Findings Examples (background, beliefs/understandings, attitudes, practices, changes): Peggy: not yet professionally trained, rather confused about fundamental concepts (eg. formative vs continuous assessment, training vs teaching), feeling totally overloaded, very uncertain, constantly seeking reassurance, has become more rather than less anxious over the last six months, needs much stronger support.

31. Findings Steven: qualified, but fairly “traditional”, positive about direction of change, but focus of “teaching” on logistics, workload and time issues, still strongly influenced by current practice (eg. video-records show SBA group interactions still very stilted and unnatural, students encouraged to use memorized phrases and reliant on notes), not convinced he is “doing it properly”, but quick to change when has opportunities to discuss and compare with peers.

32. Findings Mandy: well qualified and experienced, very enthusiastic and creative, video-records show students confident and engaged, activities tailored to suit specific context, feels positive about student development, main concerns are about lack of systematic and school-wide approach, and misunderstandings of colleagues and in community, and the effect on students of such uncertainty.

33. Findings • All teacher informants concur that the aims of assessment proposed by EMB are good for learning and teaching, but they think that their school’s assessment programme is not effective enough in meeting the new demands, schools and community still preoccupied with ‘recording data about students’ progress for ranking/comparative purposes, including for promotion’. • Teachers argue that schools have put too much emphasis on this aim, at the expense of assessment for learning

34. The need for more school-level support Planning: “The assessment activities are not well-planned and coordinated” “From 4 teachers have never met together – there is no common planning time” Provision of feedback: “Students cannot get detailed individual comments about their own performance” “Only marks or grades are given to the students. Teachers don't have time to talk to students individually”

35. The need for more school-level support The need for more formative assessment: “Currently, almost all assessment activities are traditional and summative. They can give students some feedback on their progress but not quite effective in helping them to develop” “Formative assessment tends to be neglected by both students and parents as they only believe studying for the exam is the ultimate goal of learning”

36. The need for more school-level support The need for (much) more whole school understanding and collaboration: “I can try out some new methods in my own class provided that they don't interfere too much with the normal curriculum” “It's hard. Many of these policy and practices are old and traditional. Unless I get most of my colleagues to agree to the new change, it will be difficult” “The school is pre-dominantly composed of teachers who do not strongly believe in the effectiveness of assessment policy and practice. I'm not a heavy weight in the English panel”

37. The need for more school-level support The need for (much) more whole school understanding and collaboration: “The management level of our school should update and equip themselves with policy for whole school assessment and to what extent SBA be incorporated into the curriculum. “More seminars should be given to school authority so that we don't have to fight for support when carrying out the new assessment policy” “There is not much an individual can do”

38. Conclusions Need to continue to negotiate ways to develop assessment readiness which take into account the school community, not just the teacher, as the key unit of change, by: • Reconceptualizing assessment not as a top-down change imposed upon schools, but as a process of dialogue between teachers, learners and the wider school community.

39. Conclusions • Providing teachers and schools with the opportunity to share evolving assessment beliefs and practices in order to develop a sense of ownership, common understanding of the assessment process, and a more critical but informed perspective on practice: “Fundamental changes in school assessment practices need to be planned, discussed, shared, negotiated and agreed by all teachers in each school. A corresponding assessment policy and mechanisms to bring it about need to be worked out at whole-school and classroom levels … parents (need to be informed) of the rationale underpinning the change in assessment practices…” (CDC, 2003)

40. Conclusions • Providing opportunities for teachers, students, parents and administrators to experience how assessment for learning “looks and feels very different” (Leung, 2002) to traditional exam-oriented assessments of learning.

41. Conclusions • Acknowledging that for most teachers understanding of theory only emerges through practice, and practice must not only be theoretically sound, but also totally practical, perhaps the greatest challenge of all: “At the heart of change for most teachers is the issue of whether it is practical … in the ethic of practicality amongst teachers is a powerful sense of what works and what doesn’t, of which changes will go and which will not – not in the abstract, or even as a general rule , but for this teacher in this context”. (Hargreaves, 1994: 12)