Using NAEP Assessment Items Instructionally: Teaching Students to Be Strategic

1. Using NAEP Assessment Items Instructionally: Teaching Students to Be Strategic Jeanne Foy, NAEP State Coordinator

2. Organization • Part I: Introduction to Using Assessment Items • Part II: Why Use Assessment Items for Instruction? • Part III: Web Resources • NAEP Released Items • NAEP Assessment Toolbox • Part IV: NAEP Assessment Toolbox Strategies • Reading • Math • Part V: Alaska Standards Based Assessment Resources

3. Part I • Introduction to Using Assessment Items Part I: Introduction to Using Assessment Items

4. Using Assessment Items • Wealth of resources available through released National Assessment of Educational Progress (NAEP) questions • Released items can be used instructionally • Teach to the standard • Teach students strategic reasoning skills Part I: Introduction to Using Assessment Items

5. What Is the National Assessment of Educational Progress (NAEP)? • The nation’s only nationally representative, continuous assessment of what the nation’ s students know and can do in school • Administered in Alaska as part of NCLB every other year since 2003 for • Grade 4 mathematics and reading • Grade 8 mathematics and reading Part I: Introduction to Using Assessment Items

6. Benefits of Using These Resources • Alignment with state standards/GLEs • Opportunity for students to learn state standards/GLEs • Excellent examples of assessment items • Activities that can be used to augment or supplement what teachers are already doing • Easy activities to “drop in” along with other classroom activities (graded assignments, performance-based projects, etc.) Part I: Introduction to Using Assessment Items

7. Formative Assessments • Used to gain immediate information on how students are learning and information is used to adapt instruction; students are also aware of their learning process • All activities using NAEP items are intended for formative assessments Part I: Introduction to Using Assessment Items

8. Part II • Why Use Assessment Items? Part II: Why Use Assessment Items for Instruction?

9. Teach Essential Skills “By focusing on important standards and using tools such as sample tests, teachers can help their students understand what they need to learn—and what they will be tested on.” Results Now: How We Can Achieve Unprecedented Improvements in Teaching and Learning. Alexandria, VA: Association for Supervision and Curriculum Development. Schmoker, Mike (2006). Part II: Why Use Assessment Items for Instruction?

10. Teach to the Standard • “The prevalence of higher-order standards surprises many educators but is borne out by a recent review of state assessments that found that almost all of the items on these tests—an encouraging trend—are higher-order and inferential in nature (Liben & Liben, 2005). . . .” Results Now: How We Can Achieve Unprecedented Improvements in Teaching and Learning. Alexandria, VA: Association for Supervision and Curriculum Development. Schmoker, Mike (2006). Part II: Why Use Assessment Items for Instruction?

11. Model Strategic Thinking • “. . . studies suggest that many students of diverse backgrounds are not receiving the kind of comprehension instruction that would prepare them well on assessments that are increasingly oriented toward higher level thinking with text. It is clear from research that all students need instruction in reading instruction, especially the kind that focuses on the strategies required to answer and generate challenging questions.” Raphael, T. E., & Au, K. H. (2005). QAR: Enhancing comprehension and test taking across grades and content areas. The Reading Teacher, 59 (3), 206-221. Part II: Why Use Assessment Items for Instruction?

12. Value of NAEP “. . . to many educators and policymakers, NAEP represents the gold standard in testing for its ability to assess both content and critical thinking.” — Patte Barth, Director of the Center for Public Education Part II: Why Use Assessment Items for Instruction?

13. Items Are Meant for Learning Activities Ethical test preparation practices do not include “providing students with extended practice on old or parallel forms of the test without guided practice on how to improve” Classroom Assessment for Student Learning: Doing It Right—Using It Well. Portland, OR: Educational Testing Service. Stiggins, R. J., Arter, J. A., Chappius, J., & Chappius, S. (2006) Part II: Why Use Assessment Items for Instruction?

14. Part III • Web Resources • NAEP Released Items Aligned to State Standards • NAEP Assessment Toolbox for Teachers Part III: Web Resources

15. For NAEP teacher resources, go to state assessment web page: www.eed.state.ak.us/tls/assessment/ Part III: Web Resources

16. Teacher NAEP resources on Alaska NAEP web page: http://www.eed.state.ak.us/tls/assessment/naep.html Part III: Web Resources

17. Bank of NAEP Items Linked to Alaska GLEs Part I: Introduction to Using Assessment Items

18. Table of Contents for Each Math and Reading Strand Table shows type of question Quick description of question Performance data for Alaska students Part III: Web Resources

19. NAEP Questions Linked to Alaska GLEs Posted in Word Part III: Web Resources

20. What Accompanies NAEP Assessment Items? • Multiple-choice and constructed-response questions that have been field tested • Score guides for constructed-response questions • Tables identifying GLEs assessed by each item • Student exemplars for every score level for constructed-response questions • Student performance data Part III: Web Resources

21. A note on multiple-choice questions A common misunderstanding is that multiple-choice questions cannot assess reasoning proficiency. Patterns of reasoning such as comparative reasoning and various types of inference (generalizing, author’s purpose, main idea) can be assessed in selected response format Classroom Assessment for Student Learning: Doing It Right—Using It Well. Portland, OR: Educational Testing Service. Stiggins, R. J., Arter, J. A., Chappius, J., & Chappius, S. (2006) Part III: Web Resources

22. NAEP Assessment Toolbox for Teachers • Illustrates how items can be used instructionally • Worksheet format • Models state standards • Overall objective: short, guided practice in strategic thinking Part III: Web Resources

23. Part IV: • NAEP Assessment Toolbox Strategies Part IV: NAEP Assessment Toolbox Strategies

24. NAEP Assessment Toolbox Strategies • Reading—focus on reasoning strategies and ability to evaluate/self-assess • Mathematics—students’ pattern of reasoning and errors, communication of mathematics vocabulary and concepts Part IV: NAEP Assessment Toolbox Strategies

25. NAEP Data Explanation (p. 3*) *Page numbers in headings refer to Assessment Toolbox pages Part IV: NAEP Assessment Toolbox Strategies

26. Improve Students’ Ability to Evaluate Quality of Work • Teaching students to be able to assess the quality of work is key for students to learn how to revise their own work. • Using NAEP Constructed-Response Questions and Scoring Guides To Identify Acceptable and Unacceptable Responses to Questions on a Reading Passage • The following worksheets on “Watch Out for Wombats,” from a grade 4 NAEP reading assessment show how NAEP student samples of work and NAEP scoring guides can be used to help students identify acceptable and unacceptable answers to reading questions. Part IV: NAEP Assessment Toolbox Strategies/Reading

27. Overview of Questions (p. 9) Part IV: NAEP Assessment Toolbox Strategies/Reading

28. Reading Passage (p. 11) Part IV: NAEP Assessment Toolbox Strategies/Reading

29. Student Self-Assessment Worksheet (p. 13) Part IV: NAEP Assessment Toolbox Strategies/Reading

30. Answer Key (p. 15) Part IV: NAEP Assessment Toolbox Strategies/Reading

31. Variations (covered in introduction to section on pages 7 & 8) • Discuss only one or two questions at a time or whatever seems appropriate • Give students only unacceptable responses to revise to make them acceptable • Classroom discussion and scoring of responses Part IV: NAEP Assessment Toolbox Strategies/Reading

32. Guided Practice/Reasoning Strategies Introduction (p. 59) • Released NAEP questions offer teachers many opportunities to use guided practice with students to show how to use reasoning strategies to answer questions over a reading passage. Part IV: NAEP Assessment Toolbox Strategies/Reading

33. Reasoning Strategies • National Reading Panel findings on Teacher Preparation and Comprehension Strategies Instruction conclude that “reading comprehension can be improved by teaching students to use specific cognitive strategies or to reason strategically when they encounter barriers to comprehension when reading.” Part IV: NAEP Assessment Toolbox Strategies/Reading

34. Reading Panel Findings • Comprehension strategies include a teacher guiding the reader or modeling for the reader the actions that the reader can take to enhance the comprehension processes used during reading and the reader practicing those strategies with the teacher assisting until the reader achieves a gradual internalization and independent mastery of those processes. Part IV: NAEP Assessment Toolbox Strategies/Reading

35. Procedures for Modeling ReasoningStrategies (p. 59) • Print the reading passage and questions. Ask students to complete these on their own. • After students answer the questions on their own, share the item map and reasoning strategies worksheets with students. • Use the reasoning strategies worksheets to model how to interpret and answer each question on the item map correctly with the entire class. These worksheets address the following questions: • What does the item map show about this item? • How do you know your answer is correct? • Why are the other responses wrong or incomplete? Part IV: NAEP Assessment Toolbox Strategies/Reading

36. Reading Passage (p. 63) Part IV: NAEP Assessment Toolbox Strategies/Reading

37. Questions on Passage (p. 63-65) Part IV: NAEP Assessment Toolbox Strategies/Reading

38. NAEP Item Map (p. 67) Part IV: NAEP Assessment Toolbox Strategies/Reading

39. Student Worksheet (p. 73) Students asked to rephrase GLE National data and Alaska data Percentage of students who chose each possible answer Part IV: NAEP Assessment Toolbox Strategies/Reading

40. Questions for Class Discussion (p. 74) Part IV: NAEP Assessment Toolbox Strategies/Reading

41. Constructed-Response Question Discussion (p. 69 & 70) Part IV: NAEP Assessment Toolbox Strategies/Reading

42. Scorer’s commentary for NAEP questions reinforce that students must support their opinion with evidence from the text. Part IV: NAEP Assessment Toolbox Strategies/Reading

43. NAEP Grade Levels • NAEP assessments given at three grade levels: 4, 8, and 12 • Assessment Toolbox has grade 4 and grade 8 assessment activities • For use in other grades, teachers can gauge difficulty of questions by performance data Part IV: NAEP Assessment Toolbox Strategies/Reading

44. Available NAEP Reading Questions • NAEP has many constructed-response questions • NAEP uses a variety of fiction and nonfiction • NAEP performance data can serve as benchmark for comparison on how students are doing, both to other students in Alaska and nationally Part IV: NAEP Assessment Toolbox Strategies/Reading

45. Discussion Questions • How can assessment items be used for cooperative learning? • How can assessment items be used to involve students in their own learning? Part IV: NAEP Assessment Toolbox Strategies/Reading

46. Examining Math Understanding (p.116) • Examples of how to use NAEP multiple-choice questions for one math strand each at grade 4 and grade 8 • Multiple-choice distractors carefully designed; must be plausible indicators of student thinking • Peformance data shows how common “mathematical misconceptions” are Part IV: NAEP Assessment Toolbox Strategies/Math

47. Grade 8 F&R Question (p. 131) GLE: F&R-5 translating a written phrase to an algebraic expression • 8. If n represents an even number greater than 2, what is the next larger even number?  A)  n + 1 B)  2n + 1 C)  2n D)  n + 2 E)  n2 Part IV: NAEP Assessment Toolbox Strategies/Math

48. Mathematical Error Revealed by Each Wrong Response (p. 132) A: This expression would create an odd number. B: This expression would create an odd number; also, 2n would not create the next larger even number. C: This would create an even number, but not the next larger even number. D: Correct answer E: This would create an even number, but not the next larger even number. Part IV: NAEP Assessment Toolbox Strategies/Math

49. Checking for Understanding by Using Response Cards • Low-cost, available materials • Technique that can be used for any content, for open-ended questions or multiple-choice questions • Model strategies to address performance gap for “nonroutine” problems Part IV: NAEP Assessment Toolbox Strategies/Math

50. “Moment of Contingency” • “To gauge the understanding of the whole class, the teacher needs to get responses from all the students in real time. • One way to do this is to have all students write their answers on individual dry-erase boards, which they hold up at the teacher's request. The teacher can then scan responses for novel solutions as well as misconceptions.” Leahy, S., Lyon, C., Thompson, M., & Wiliam, D. (2005). Classroom Assessment: Minute by Minute, Day by Day [Electronic version]. Educational Leadership: Assessment to Promote Learning 63 (30, 19-24) Part IV: NAEP Assessment Toolbox Strategies/Math