Using Assessment Program Resources to Shape Effective ELA Instruction

1. Using Assessment Program Resources to Shape Effective ELA Instruction Amy F. Radikas, Literacy Specialist Renee A. Savoie, NAEP State Coordinator Connecticut Assessment Forum III Crowne Plaza, Cromwell August 13, 2012

2. Session Overview • Key shifts in ELA/Literacy • NAEP and PISA basics • Evidence of the shifts illustrated through NAEP & PISA • Discussion

3. CCSS Requires Three Shifts in ELA/Literacy #1 Building knowledge through content-rich nonfiction #2 Reading, writing and speaking grounded in evidence from text,both literary and informational #3 Regular practice with complex texts and its academic language achievethecore.org

4. ACT Report In 2006, ACT released a report called Reading Between the Lines. The findings suggested that the ability to read complex texts is the “clearest differentiator between those ready for college-level reading and those not.”

5. ACT: Reading Between the Lines The most important implication of this study: “What students could read, in terms of its complexity, was at least as important as what they could do with what they read.” CCSS Appendix A. p. 2

6. What is NAEP? • National Assessment of Educational Progress • Nation’s Report Card • Established by Congress in 1969 to answer questions about student academic performance • The Nation’s ONLY monitor of what students know and can do • “common yardstick” • “Survey” design • National, regional and state-level results

7. Overall NAEP Reading Results for Connecticut 2003-2011 * indicates a statistically significant difference when compared to performance in 2011.

8. Grade 4 Reading in 2011: Average Scale Score Comparisons Higher avg. ss. than CT Avg. ss. not significantly different from CT Lower avg. ss. than CT

9. Grade 8 Reading in 2011: Average Scale Score Comparisons Grade 8 Reading in 2011: Average Scale Score Comparisons Higher avg. ss. than CT Avg. ss. not significantly different from CT Lower avg. ss. than CT

10. What is PISA? Click to view YouTube video about PISA • Program for International Student Assessment (sponsored by the OECD) • Administered every 3 years since 2000 • Assesses 15-year-old students • Measures application of knowledge and skills to problems in a real-life context

11. PISA aims to answer… • Are students well prepared to meet the challenges of the future? • Are they able to analyze, reason, and communicate their ideas effectively? • Do they have the capacity to continue learning throughout life?

12. Who Participates in PISA? OECD countries (34) Non-OECD education systems (33)

13. PISA 2009 Reading Literacy Results • Data for 65 education systems (34 OECD and 31 non-OECD) • U.S. not measurably different from OECD average • U.S. scored below 9 education systems • U.S. scored above 39 education systems

14. PISA Reading Literacy Trends

15. PISA and NAEP: Key Differences • PISA includes a considerable amount of noncontinuous text. • NAEP measures “meaning vocabulary.” • NAEP Grade 8 and 12 passages are notably longer than PISA passages. • In terms of readability and grade level, PISA is closer to Grade 12 NAEP.

16. The Origins of CCSS CCSS International Assessments NAEP

17. CCSS Requires Three Shifts in ELA/Literacy #1—Building knowledge through content-rich nonfiction #2—Reading, writing and speaking grounded in evidence from text,both literary and informational #3—Regular practice with complex texts and its academic language achievethecore.org

18. Informational Text • Students are required to read very little informational text in elementary and middle school. • Informational text is more difficult for students to comprehend than narrative text. • Informational text makes up the vast majority of required reading in college/workplace.

19. NAEP Reading: Passage Types by Grade CCSS ELA p.5

20. In the Early Years Children must: • Develop strong foundational cognitive skills (literacy/communication).

21. Grade Two • Informational texts use clear and consistent formats, use of simple headings to organize information into categories, illustrations extend the meaning, and simple graphics support understanding of content • Elfrieda H. Hiebert,Fundamentals of Literacy Instruction and Assessment, Pre-K–6Edited by Martha C. Hougen, Ph.D., & Susan M. Smart

22. An Example from NAEP • NAEP 2011 Grade 4 Informational Passage: Marian’s Revolution by SudiptaBardhan-Quallen Available through the NAEP Questions Tool

23. One of 10 Released Items Why is "A Voice for Civil Rights" a good heading for the section that follows it on page 3? Use information from the article to support your answer.

24. Student Responses: Full Comprehension

25. Student Responses: Partial Comprehension

26. Student Responses: Little or No Comprehension

27. Item-Level Performance Data NP= National Public ‡ Reporting standards not met.

28. CCSS Requires Three Shifts in ELA/Literacy #1—Building knowledge through content-rich nonfiction #2—Reading, writing and speaking grounded in evidence from text,both literary and informational #3—Regular practice with complex texts and its academic language achievethecore.org

29. Evidence to Support Text Dependent Questions • Can be literal (checking for understanding) but must also involve analysis, synthesis, evaluation. • Focus on word, sentence, and paragraph, as well as larger ideas, themes, or events. • Focus on difficult portions of text in order to enhance reading proficiency. • Can also include prompts for writing (DOK)and discussion questions. achievethecore.org

30. Bloom’s Taxonomy [1956 ] & Bloom’s Cognitive Process Dimensions [2005] Karin Hess

31. Webb’s Depth-of-Knowledge Levels • DOK-1 – Recall & Reproduction - Recall of a fact, term, principle, concept, or perform a routine procedure • DOK-2 - Basic Application of Skills/Concepts - Use of information, conceptual knowledge, select appropriate procedures for a task, two or more steps with decision points along the way, routine problems, organize/display data, interpret/use simple graphs • DOK-3 - Strategic Thinking - Requires reasoning, developing a plan or sequence of steps to approach problem; requires some decision making and justification; abstract, complex, or non-routine; often more than one possible answer • DOK-4 - Extended Thinking - An investigation or application to real world; requires time to research, problem solve, and process multiple conditions of the problem or task; non-routine manipulations, across disciplines/content areas/multiple sources Karin Hess

32. The Hess Cognitive Rigor Matrix integratesBloom+ Webb Different models used to describe cognitive rigor: • Bloom– What type of thinking (verbs) is needed to complete a task? • Webb – How deeply do you have to understand the content to successfully interact with it? How complex is the content? Karin Hess

33. The Hess Cognitive Rigor Matrix Applies Webb’s DOK toBloom’s Cognitive Process Dimensions

34. DOK is about complexity—not difficulty! • The intended student learning outcome determines the DOK level. What mental processing must occur? • While verbs may appear to point to a DOK level, it is what comes after the verb that is the best indicator of the rigor/DOK level. • Describe the process of photosynthesis. • Describe how the two political parties are alike and different. • Describe the most significant effect of WWII on the nations of Europe. Karin Hess

35. Let’s practiceYour class has just read some version of Little Red Riding Hood. What is a basic comprehension question you might ask?What is a more rigorous question you might ask? Karin Hess

37. How can we apply these ideas back in our schools AND CLASSROOMS?

38. Guiding Questions • What skills & concepts are most important? • Is the intended rigor of skills/concepts reflected in materials used (e.g., texts, tasks) • Is there a range of DOK (rigor) within the learning activities/lesson?

39. Standard Ten Standard One Standards Two through Nine Increasing Range and Complexity Increased Ability to Use Text Evidence achievethecore.org

40. PISA Reading Unit 3: Graffiti See Page 13

41. Question 4: Graffiti (p. 17) • Full Credit Guidelines: Explain opinion with reference to style or form of one or both letters. Refers to criteria such as structure of argument, cogency of argument, strategies for persuading readers, etc. • Example: “Helga’s. She gave you lots of different points to consider and she mentioned the environmental damage that graffiti artists do which I think is very important.”

42. Question 4: International Performance Data

43. CCSS Requires Three Shifts in ELA/Literacy #1—Building knowledge through content-rich nonfiction #2—Reading, writing and speaking grounded in evidence from text,both literary and informational #3—Regular practice with complex texts and its academic language achievethecore.org

44. Academic Vocabulary • Words that add to students’ language ability (e.g., maintain, fortunate, required, tend, contrast/compare, insisted, summarize) • Words that are needed in a content area (e.g., isotope, peninsula, photosynthesis, cubism, isosceles triangle)

45. More Examples from NAEP • NAEP 2009 Grade 12 Literary Passage: The Open Window by Saki Available through the NAEP Questions Tool

46. NAEP’s “Meaning Vocabulary” • On page 3, the narrator says Framton has the delusion that total strangers are interested in his nervous condition. This means Framton • has a common illness related to his nerves. • is worried about what people think of him. • dreams that he is constantly being watched. • has a mistaken idea about how others view him.

47. Grade 8 Example On page 2, when the author of the biographical sketch says that E. B. White's essays "appealed to an urbane crowd," he is describing the audience for White's writing as • having sophisticated tastes. • leading busy lives. • being politically conservative. • having a good sense of humor.

48. The Origins of CCSS CCSS International Assessments NAEP

49. Discussion • What are the opportunities and challenges related to the shifts? • How can districts support teachers in making these important shifts? • Revisit your definition of rigor – has it changed? In what ways? • What existing curriculum, assessment materials, or activities will you examine for a range of cognitive rigor? • What types of supports and resources can CSDE provide?

50. Related Resources Many papers and presentations available at www.nciea.org or contact Karin Hess khess@nciea.org • Hess, K. (2004). “Applying Webb’s Depth-of-Knowledge (DOK) Levels in social studies and science” [online]: http://www.nciea.org/publications/DOKsocialstudies_KH08.pdf http://www.nciea.org/publications/DOKscience_KH11.pdf • Little Red Riding Hood handout http://www.nciea.org/publications/Little%20Red%20Riding%20Hood-CRM_KH11.pdf • Achievethecore.org—Student Achievement Partners provides modules for professional development and other valuable CCSS resources.