Classroom Assessment of Reading and Listening

1. Classroom Assessment of Reading and Listening Dr. Liying Cheng Faculty of Education, Queen’s University chengl@educ.queensu.ca

2. Overview • Integration of language skills • Learning targets • Interrelationship of assessment and instruction • Assessment methods and procedures

3. Agenda Review • The relationship between assessment and instruction Subject matter - English • Understanding what the listening process – the input of listening material • Understanding what the reading process – the input of reading material Assessment strategies • Work with the purposes, types, and use of classroom assessment • Portfolio – self- & peer-assessment; feedback & conference

4. Developing and Choosing Methods for Assessment

5. Learning targets • Indicate 1) what a student is to know and/or do as a result of instruction (content) and 2) the criteria for evaluating performance (criteria) • Knowledge and simple understanding • Declarative (know what) • Procedural (know how) • Deep understanding and reasoning • Skills • Product • Affect

6. Classroom assessment types • selected-response • Supply-response (constructed res.) • performance-based tasks • teacher oral questioning/observations • self-report (self/peer-assessment) Assessment for Learning OR Assessment of learning?

7. Key Relationship • Write down the name of a course you recently taught and the kind of students you had • Write down up to three instruction goals - What did you wish your students to achieve by the end of the course? Or what language proficiency level to achieve and in what sub-skills? • Write down your assessment purposes • Write down the assessment methods used to achieve these three goals in accordance to your assessment purposes – what methods have you used to assess your students?

8. Activities What is learning? • Listening 1 • Listening 2 • Listening 3 What is reading? • Reading 1 • Reading 2 • Reading 3

9. Listening Theoretical perspectives by Tony Lynch (1998) Listening process • Speech recognition • Memory in processing • Discourse comprehension • Accessing the process

10. Listening Context in listening • Setting • Behavioral • Extra-situational Factors influencing listening • Text factors • Listener factors

11. Listening Influencing factors by David Mendelshon (1998) • Text characteristics • Listener characteristics • Interlocutor characteristics • Task characteristics • Process characteristics

12. Listening – Vandergrift (2004) • Learning to listen • Listening to learn • Empirical research on listening instruction • Advance organizers • Visual support • Captions • Vocabulary knowledge (bottom-up skills) • Listening strategies (Metacognitive awareness)

13. Teaching of listening by David Mendelsohn (1998) • A strategies-based approach • Skills integration • Audio/video and non-verbal element • Technology (computer-assisted) • Interactive Listening (bidirectional listening) • Academic Listening • Narrow listening (Vandergrift, 2004) • Listening to train the ear (Vandergrift, 2004)

14. Reading models (1) Gough (1972) proposes what may be classified as a phonics-based or "bottom-up" model of the reading process which portrays processing in reading as proceeding in serial fashion, from letters to sounds, to words, to meaning, in the progression suggested in the accompanying figure.

15. Reading model (2) • The flow of information proceeds from the top downward so that the process of word identification is dependent upon meaning first. The figure depicts fluent readers as actively engaged in predicting or hypothesis-testing when progressing through text - "psycholinguistic guessing game" (Goodman, 1970).

16. Reading model (3) • Rumelhart (1977) proposes an interactive model in which both letter features or data-driven sensory information and nonsensory information come together at one place.In this model reading is not viewed simply as either a "bottom-up" or a "top-down" process, but instead as a synthesizing of patterns, calling for the application or integration of all of the previously identified knowledge sources shown in the accompanying figure.

17. Reading

18. Reading process by Thom Hudson (1998) • Interpreting letters and words • Interaction between readers and texts • L2 reading • Transferring of L1 literacy to L2 • L1 difference • Genre difference

19. Instructional implications • Ensure fluency in word recognition • Emphasize the learning of vocabulary • Activate background knowledge • Ensure acquisition of linguistic knowledge and general comprehension • Recognition pf text structures and discourse organization • Promote development of strategic readers than mechanical application of strategy checklists • Build reading fluency and rate • Promote extensive reading • Develop intrinsic motivation for reading • A coherent curriculum William Grabe (2004)

20. Teaching of readingby Bamford & Day (1998) • Grammar translation • Comprehensive questions and language work • Skills and strategies • Extensive reading • Automaticity of word recognition • Affect • Social-cultural factors • The power of extensive reading

21. Classroom assessment strategies Purposes of assessment • Supporting students’ learning • Monitoring student progress in the classroom • Sorting and selecting students • Program evaluation/accountability

22. Classroom assessment strategies Methods to assess listening and reading • Portfolio • Reading/listening logs • Book report • ? • ?

23. Activate prior knowledge • Brainstorm all the different questions, concerns, or information needs that a teacher may have about groups of students or individual students at a given time. • Prior learning experience • Current knowledge • Interests • ? • ?

24. Learning the information 2. Record corresponding students’ learning needs and targets (swap charts) 3. Identify methods to collect the above information if not available (swap charts) 4. Reorganize your original chart by answering the following questions • What might be the intended or unintended effects of the different assessment methods or sources of information on students before, during, or after instruction? • Which points in time (before, during, or after) place greater demands upon the assessment methods in terms of reliability and validity? Explain.

25. Further questions • Which assessment methods demand greater evidence of reliability and validity compared with other methods? Explain • What effects would lack of reliability and validity have on the uses of assessment information at each point in time? Are the consequences of low reliability or validity of assessment methods greater when assessing students before, during, or after instruction? • Is it appropriate to grade all assessments gathered before, during, or after instruction? Why or why not? • How are assessment purposes and grading related? Which source(s) of information are most practical and efficient for each information need?

26. Key components of classroom assessment • Formative vs. summative assessment • Providing feedback (corrective feedback) • Observations • Questioning • On-going assessment

27. Formative/summative assessment • FA - To provide ongoing feedback to the teacher and students – the improvement of student motivation and learning (see Figure 5.1) • SA- To document student performance after the instruction

28. Feedback • Relate performance to targets/standards • Indicate progress • Indicate corrective procedures • is given frequently and immediately • is specific and descriptive • Focuses on key errors

29. Informal (informed) observation • Non-verbal behaviors • Facial expressions • Body language • Gestures • Assessing voiced-related cues • Sources of errors • Tone of voice, Loudness, Intensity, Pauses, Silence, Voice level, Inflection, word spacing, emphases etc.

30. Characteristics of questioning • State questions clearly and succinctly • Match questions with learning targets • Involve the whole class • Allow sufficient wait time for student responses • Give appropriate responses to students answers • Avoid questions answered by yes or no • Probe initial responses when appropriate • Avoid tugging, guessing, and leading questions • Avoid asking students what they think they know • Ask questions in an appropriate sequence

31. Ongoing Assessment • Ongoing assessment has to do with learning activity that occurs continuously. • It has less to do with written reports and far more to do with the interactive, dynamic roles of both teachers and learners. • It has to do with responding to learners' questions every day and with actively noting the kinds of questions learners ask, the ways in which learners respond to print and oral communications, the kinds of mistakes they make, the ways in which they go about correcting their own mistakes, and the ways in which [others] might correct them. • This kind of ongoing observation and assessment is inseparable from good teaching practice. • Adventures in Assessment, Volume 2.

32. The ‘what’ of our assessment The importance of assessment/testing constructs • listening • Reading What can we learn from other contexts?

33. Hong Kong Secondary Schools • School-based assessment • Read • Listen • Speak/writing

34. The Ontario Secondary School Literacy Test in Canada Reading constructs - a total of 12 short selections in three different text types: • information (e.g., explanation, opinion) 50%; • graphic (e.g., graph, schedule, instructions) 25%; • narrative (e.g., story, dialogue) 25%. in three test formats: • multiple-choice (40 questions); • constructed response (35 questions); • constructed responses requiring an explanation (25 questions).

35. The Ontario Secondary School Literacy Test • three reading skills: • understanding directly stated ideas and information; • understanding indirectly stated ideas and information; • making connections between personal experiences and ideas and information in a selection. • Four reading strategies: • vocabulary, • syntax, • organization, • graphic features.

36. Scoring Criteria (Reading) • Multiple choice items are scored separately. • Constructed Response (CR) questions ask students to respond in a few words, and answers are marked correct (1 point) or incorrect (0 point). • Constructed Response with Explanation (CRE) questions ask the students to justify or explain the thinking behind their answers. And the answers are marked correct (1 point), partly correct (0.5 point) or incorrect (0 point)

37. The Ontario Secondary School Literacy Test in Canada Writing - four writing tasks: • a summary, • a series of paragraphs expressing an opinion, • a news report, and • an information paragraph. Students need to pass both components of the OSSLT.

38. Scoring Criteria (writing) • Two steps in the marking of writing--holistic scoring and analytical scoring. • Holistic scoring (0-4 points) • If the student performances do not meet the requirements to pass, then all four writing tasks will go on to step 2: analytic scoring, evaluated by four separate characteristics: • Main idea • supporting details, • organization, • Spelling, grammar and punctuation.

39. What is portfolio? • A portfolio is a purposeful, systemic and well-organized collection of sample student work that exhibits the student’s efforts, progress, and achievement in one or more areas with pre-established guidelines. • The collection must include student participation in selecting contents, the criteria for selection, the criteria for judging merit, and evidence of student reflection with progress documented and conferences held between the teacher and the students (Arter & Spandel, 1992)

40. Why use portfolio? (1) • Portfolio gives students the opportunity to direct their own learning. • Portfolio can be used to determine students’ level of achievement. • Portfolio can be used to determined students’ growth over time.

41. Why use portfolio? (2) • Portfolio can be used to understand how students think, reason, organize, investigate and communicate. • Portfolio can be used to communicate students’ efforts, progress toward accomplishing learning goals, and accomplishments.

42. Types vs. purposes of portfolio

43. Entries of portfolio (1) • Table of contents: Including the title and the category of each portfolio entry, types of listening/reading materials, and the page number in the portfolio. • Diagnostic self-evaluation: Writing a reflective piece on the learning progress both at the beginning and the end of the term • Reflection of listening/reading logs: documenting what have been listen/read (accent, topics etc…) • Submitting one log on the last Friday of each month according to what they have listened/read

44. Entries of portfolio (2) • Self-monitored listening/reading – self-assessment • Shared listening/reading – peer assessment (enjoyable) • Any follow-up activities: • Writing a piece (argumentative or reflective) on what listened/read; or • Holding a group discussion in/outside class (integration of skills)

45. Entries of portfolio (3) • Radio, TV, MTV, films etc… • Newspapers, journal articles, short stories, novels, radio, films etc. • Academic genres of reading from the following 3 categories: narration, exposition, and argumentation. • Leisure listening:

46. Marking of portfolio: Self-evaluation • What is the strength/ weakness of this piece? • What did I learn while listening/reading this piece? • What would I do differently if I were to listen/read this again? • What was the most difficult aspect of listening/reading this piece? • How would I rank this piece in term of difficulty or enjoyment on a scale of 1 to 5 (5 is the highest)? • Whom did I ask to share this listening/reading piece? • What suggestions did he or she make? Were they helpful or not? Why?

47. Marking of portfolio: peer-assessment Describe who you are as a listener/reader:____________ • What is the strength of this piece? • Does the beginning attract your attention? • Does the speaker/writer provide evidence to support what is claimed? • Is the supporting detail effective to support the speaker’s/writer’s point? • Are there any parts you had difficulty understanding? • What would you suggest that would improve the speaking/writing?

48. Provide on-going feedback • Student-teacher conference • Hong Kong - SBA

49. Communicating portfolio results In student-led conference or at a class meeting, all the portfolios can be displayed to provide an opportunity for students to share their portfolios with their classmates (Tierney et al., 1991) and to learn from each other.

50. Communicating portfolio results • In parent-teacher conference, portfolios offer an excellent means for parents to enter into the classroom through reading their children’s portfolios, which “gives them a more intimate basis for seeing aspects of their children’s experiences in school”, provides a framework for meaningful discussions of the students’ achievements, progress, and areas to work on next (Linn & Gronlund, 2000, p. 311), and eventually invites them actively involved in enhancing education.