1 / 48

Interrelationship between Assessment and Instruction

Interrelationship between Assessment and Instruction. Dr. Liying Cheng Faculty of Education, Queen’s University chengl@educ.queensu.ca. Overview. How we define the interrelationship between assessment and instruction How we define learning and learning targets

holden
Download Presentation

Interrelationship between Assessment and Instruction

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Interrelationship between Assessment and Instruction Dr. Liying Cheng Faculty of Education, Queen’s University chengl@educ.queensu.ca

  2. Overview • How we define the interrelationship between assessment and instruction • How we define learning and learning targets • Key components of classroom assessment • Project 1 – teacher classroom assessment • Project 2 – large-scale testing and second language students

  3. AEG – Faculty of Education, Queen’s U • Assessment and Evaluation Group (AEG) is an evaluation research and consulting group at the Faculty of Education, Queen’s University. AEG operates in many contexts, at many levels, with many methods of inquiry all directed at the improvement of learning. • I acknowledge the use of certain slides from my colleagues – Don Klinger and Lyn Shulha – in this seminar.

  4. Rationale: Key relationship Teaching Learning Assessment

  5. Interrelationship • Impact of assessment (testing and public examinations) on classroom instruction (teaching and learning) – washback research (Latham, 1877; Li, 1990; Alderson & Wall, 1993; Cheng, Watanabe, with Curtis, 2004)

  6. Interrelationship • Links between assessment and instruction in terms of authenticity and congruence of assessment practices in relation to a program of study (Douglas, 2000; Bachman & Cohen, 1998) • Assessment of student outcomes in term of curriculum and teaching (Johnston, 2000)

  7. Interrelationship • Teacher assessment where the teacher is the agent of assessment - conducting both formative and summative assessment – linking to the purposes of assessment (Brindley, 2001; Brindley, 2007; Cheng et. al. 2004; Rea-Dickins, 2004).

  8. What is Learning? A curricular definition Learning - acquiring knowledge, skills, attitudes, and values and experiences. A cognitive definition Learning - a process of formulating new and more complex understandings of the world Learning - revising and refining mental constructs, i.e., the understandings that guide how we think, speak and behave

  9. Wilson’s Taxonomy for Teachers Novice • A journey from novice to expert • Helps you monitor how students are thinking & performing • Cues you about the kinds of questions or activities that can help students think more deeply about their learning? I.C.E. Ideas • Connections • Within content • To self Extensions Other Contexts Expert

  10. Learning targets (McMillan, 2004) • Indicate 1) what a student is to know and/or do as a result of instruction and 2) the criteria for evaluating the performance (criteria) • Knowledge and simple understanding • Declarative (know what) • Procedural (know how) • Deep understanding and reasoning • Skill • Product • Affect

  11. Classroom assessment • The collection, evaluation, and the use of information to help teachers make better decisions that improve student learning. • Classroom assessment is more than testing and measurement. • The fours essential components to implementation classroom assessment are • Purpose • Measurement • Evaluation • Use

  12. Classroom Assessments • Developing and Choosing Methods for Assessment • Collecting Assessment Information • Judging and Scoring Student Performance • Summarizing and Interpreting Results • Reporting Assessment Findings Principles for Fair Student Assessment Practices for Education in Canada (1993)

  13. TEACH and e-Learning grants • Classroom Assessment: Theory and Practice • Computer-Based Assessment Development • Researching Teaching English as a Second Language

  14. Classroom Assessment

  15. B Ed Instructors • The instructors commented much of their time with the B Ed candidates will be spent exploring • practicum expectations, • professional conduct, • classroom procedures and routines, • shaping instruction for individual, small group and whole class activities, • motivating students, • how to approach issues in classroom management, and • Emphasize issues in assessment and evaluation.

  16. Associate Teachers • All of the associate teachers suggested that candidates need some familiarity with • rubrics, • making judgments about student learning, • achievement chart categories, and • the four levels of achievement. • Many suggested that it would be an asset to be able to create rubrics. This requires some understanding of how to differentiate a Level 2, Level 3, and Level 4 response to a learning task.

  17. School District Administrators • They expressed a strong interest in seeing B Ed candidates grapple with the concept of assessment for learning (formative assessment). • When they also motioned the need to construct grades (summative assessment), they suggested that candidates have a strong understanding of how to determine grades from a variety of instruments  and achievement charts and how to identify which of these represent most recent and consistent performance.  • Matching assessment tools to Ministry achievement charts and the ability to do performance assessment rounded out their suggestions to this year's class.  

  18. Rubric to Monitor Learning and Achievement Levels of Performance

  19. Growth in Bike Riding

  20. Classroom Assessment: Theory and Practice • The Fallibility of Assessment • What Does Learning Look Like? • Assessment of Learning • Assessment for Learning • Developing Rubrics • Developing Assessment Instruments • The Multiple Perspectives of Assessment

  21. USA has Standardized Testing No set curriculum. Separation of assessment and instruction. No attempt to involve teachers in interpreting the responses and results. Teachers administer the tests because they have to. Canada has Large-scale Assessment Provincial curriculum. Assessment are based on learning expectations and are integrated into instruction. Teachers are involved with test design, development, scoring and interpreting. Teachers administer the tests because they have to and because schools have growth plans and resources linked to assessments. Differences in Large-scale testing

  22. The Myths of Testing(Stiggins, 2004) Testing motivates students to learn • If I threaten you will fail and then you will try harder • If a little intimidation does not work, try more intimidation • Maximize anxiety to maximize learning Testing helps teachers make important instructional decisions • Students are not assessment users Important assessment decisions can be made once a year • Investment of time, effort, and money into large-scale testing supports this belief Learning how to assess is not as important as learning how to teach • Teachers teach and testing professionals test • Assessment literacy

  23. The Challenges of the Ontario Secondary School Literacy Test (OSSLT) for ESL/ELD Students Cheng, L., Klinger, D., & Zheng, Y. (2007). The challenges of the Ontario Secondary School Literacy Test for second language students. Language Testing, 24(2), 1-24.

  24. Research Context • The increasing number of English as a Second Language (ESL) and English Literacy Development (ELD) students in Ontario secondary schools. Immigration to Ontario increased 23% vs. ESL teachers and programs in Ontario schools declined 30% • Large-scale educational testing has increasingly been used to measure and ensure student competency or provide system accountability(Firestone, Mayrowetz, & Fairman, 1998; Ryan, 2002).

  25. Research Context • These large-scale tests are constructed and normed for first language (L1) English speakers. Research suggests, however, that they may have lower reliability and validity for L2 students and should be interpreted differently (Abedi, Leon, & Mirocha, 2003). • The confluence of both increased numbers of L2 students and this expanding testing framework in schools has created a new and largely unanticipated educational problem – alarmingly high failure rates of these students (Watt & Roessingh, 2001).

  26. ESL/ELD Students’ Performances in the OSSLT • Of all the eligible ESL/ELD students, only 46% in Feb. 2002, 45% in Oct. 2002, and 54% in October 2003 participated in the test administration. • The pass rate of ESL/ELD students in the last three test administrations are 37%, 34% and 42% respectively. While the total pass rate of all students are 75%, 72% and 77% respectively • ESL/ELD students were failing the Ontario Secondary School Literacy Test (OSSLT) at a rate higher than any other group of students • Pass rates: 42% vs. 77% in 2003 • Participation rates: 54% vs. 91% (EQAO, 2004)

  27. The Ontario Secondary School Literacy Test (OSSLT) “The OSSLT is a useful quality assurance measure that shows the extent to which Ontario students are meeting a common, basic standard for literacy across the province” (EQAO[1], 2002, p. 1). The OSSLT is developed by the Education Quality and Accountability Office (EQAO) - an independent agency operating within a framework established by the provincial government of Ontario. [1]We acknowledge the support from EQAO for releasing the February 2002 OSSLT data for this study.

  28. The Ontario Secondary School Literacy Test (OSSLT) • All students in public and private schools working toward an Ontario Secondary School Diploma are required to write the OSSLT in Grade 10. Students who have been eligible to write the OSSLT at least twice and have been unsuccessful at least once are eligible to fulfill the requirement through the Ontario Secondary School Literacy Course (OSSLC). Successful completion of the OSSLT or OSSLC is a graduation requirement.

  29. The Ontario Secondary School Literacy Test (Reading) Reading component - a total of 12 short selections in three different text types: • information (e.g., explanation, opinion) 50%; • graphic (e.g., graph, schedule, instructions) 25%; • narrative (e.g., story, dialogue) 25%. in three test formats: • multiple-choice (40 questions); • constructed response (35 questions); • constructed responses requiring an explanation (25 questions).

  30. The Ontario Secondary School Literacy Test (Reading) • three reading skills: • understanding directly stated ideas and information; • understanding indirectly stated ideas and information; • making connections between personal experiences and ideas and information in a selection. • Four reading strategies: • vocabulary, • syntax, • organization, • graphic features.

  31. Scoring Criteria (Reading) • Multiple choice items are scored separately. • Constructed Response (CR) questions ask students to respond in a few words, and answers are marked correct (1 point) or incorrect (0 point). • Constructed Response with Explanation (CRE) questions ask the students to justify or explain the thinking behind their answers. And the answers are marked correct (1 point), partly correct (0.5 point) or incorrect (0 point)

  32. The Ontario Secondary School Literacy Test (Writing) Writing - four writing tasks: • a summary, • a series of paragraphs expressing an opinion, • a news report, and • an information paragraph. Students need to pass both components of the OSSLT. OSSLT is administered over a two-day period.

  33. Scoring Criteria (writing) • Two steps in the marking of writing--holistic scoring and analytical scoring. • Holistic scoring (0-4 points) • If the student performances do not meet the requirements to pass, then all four writing tasks will go on to step 2: analytic scoring, evaluated by four separate characteristics: • Main idea • supporting details, • organization, • Spelling, grammar and punctuation.

  34. Research framework • Research studies have shown that the test items, the content, types and context of reading passages, and the relationships of these intervening factors can have a significant impact on students’ performance (Anderson et al., 1991; Freedle & Kostin, 1993; Kobayashi, 2002, Lee, 2002; Peretz & Shoham, 1990; Perkins, 1992). • Research into writing assessment has demonstrated that different types of writing tasks create different challenges for students (e.g., Connor-Linton, 1995a, 1995b, Kobayashi & Rinnert, 1996, Hamp-Lyons, 1996).

  35. Research questions • Compared with Non-ESL/ELD Ontario students, what is the performance of ESL/ELD students on the OSSLT? • Is there a significant difference in test performance between the two groups of students in relation to the reading types, skills and strategies, and the four writing types?

  36. Mean score for Non-ESL Ontario students#=138392 Mean score for ESL/ELD students #=2686 Reading Types: -Information (e.g., explanation, opinion) 64/90 (71%) 46/90 (51%) - Graphic (e.g., graph, schedule, instructions) 36/50 (72%) 26/50 (52%) - Narrative (e.g., story, dialogue) 47/60 (78%) 34/60 (57%) OSSLT Reading Results February 2002

  37. Reading Skills: - Understands directly stated ideas and information 46/60 (77%) 36/60 (60%) - Understands indirectly stated ideas and information 65/90 (72%) 46/90 (51%) - Makes connections between experiences and the ideas, information in the reading 34/50 (68%) 24/50 (48%) Reading Strategies: - Vocabulary 22/30 (73%) 15/30 (50%) - Syntax 19/30 (63%) 13/30 (43%) - Organization 25/36 (69%) 19/36 (53%) - Graphic features 16/24 (67%) 12/24 (50%)

  38. Mean score for Non-ESL Ontario students#=4068 Mean score for ESL/ELD students #=2686 Reading Test Formats: -Multiple choice (MC) 62/80 (78%) 47/80 (59%) - Constructed response (CR) 53/70 (76%) 38/70 (54%) - Constructed response with explanations (CRE) 31/50 (62%) 21/50 (42%)

  39. OSSLT Writing Results February 2002 Percentage of Non-ESL Ontario students#=138392 Percentage of ESL/ELD students #=2686 Summary Blank/Illegible & Irrelevant Content / Off-task (0 points) 18% 33% 1 point 3% 3% 2 points 26% 22% 3 points 36% 34% 4 points 17% 11% Paragraphs expressing an opinion (0 points) 7% 22% 1 point <1% <1% 2 points 18% 29% 3 points 50% 38% 4 points 25% 11%

  40. Percentage of Non-ESL Ontario students#=138392 Percentage of ESL/ELD students #=2686 News report (0 points) 7% 25% 1 point <1 2% 2 points 12% 20% 3 points 53% 41% 4 points 27% 12% Information paragraph (0 points) 11% 27% 1 point <1% 2% 2 points 15% 22% 3 points 50% 39% 4 points 23% 10%

  41. Discriminant Analysis • Y grouping =AB1X1B2X2B3X3 ….E • ESL status: 1 = ESL/ELD students; 2 = non-ESL/ELD students • ESL N=2688; Non-ESL N=4068 • For all the following tables, p < .000

  42. Discriminant Analysis of Reading & Writing in OSSLT Feb 2002

  43. Discriminant Analysis of Reading Types in OSSLT

  44. Discriminant Analysis of Reading Skills & Strategies in OSSLT Feb 2002

  45. Discriminant Analysis of Reading Formats in OSSLT Feb 2002

  46. Discriminant Analysis of Writing Types in OSSLT Feb 2002

  47. Significance of the Study • ESL/ELD students make up an substantial and growing portion of Canadian high school population. It is therefore important to identify factors and barriers that are specifically associated with their literacy development. • This part of the results have informed us about potential difference in ESL/ELD students’ literacy development through OSSLT. Such an information can be used to support these students’ learning.

More Related