PARCC State Educator Item Review Meeting March 2013

1. PARCC State Educator Item Review Meeting March 2013 Note: all items included in this presentation are for illustrative training purposes only. They are not representative of PARCC assessment items.

2. Overview of Training • Purpose and Features of PARCC Summative Assessments • PARCC Summative Assessments (PBA, EOY) • Your Charge as a Committee Member • Item Review Process by Core Leadership Group • Item Review Criteria by Core Leadership Group • Item Review Process and Criteria by State Educators

3. Purpose of PARCC Summative Assessments • Determine whether students are college- and career-ready (CCR) or on track to become CCR • Assess the full range of the Common Core State Standards (CCSS) for reading, writing, and language • Measure the full range of student performance, including the performance of high- and low-performing students* • Provide data for accountability, including measures of growth • Incorporate innovative approaches throughout the system

4. Features of the PARCC Summative Assessment System The PARCC summative assessment system is composed of two assessments: • Performance Based Assessment • End of Year Assessment

5. Performance-Based Assessment • Literary Analysis Task (LAT) • Literary Analysis Task plays an important role in honing students’ ability to read complex text closely, a skill that research reveals as the most significant factor differentiating college-ready from non-college-ready readers. This task will ask students to carefully consider literature worthy of close study and compose an analytic essay. • Research Simulation Task (RST) • Research Simulation Task asks students to exercise the career- and college- readiness skills of observation, deduction, and proper use and evaluation of evidence across text types. In this task, students will analyze an informational topic presented through several articles or multimedia stimuli, the first text being an anchor text that introduces the topic.  Students will engage with the texts by answering a series of questions and synthesizing information from multiple sources in order to write two analytic essays. • Narrative Task (NT) • Narrative Task broadens the way in which students may use this type of writing. Narrative writing can be used to convey experiences or events, real or imaginary. In this task, students may be asked to write a story,  detail a scientific process, write a historical account of important figures, or to describe an account of events, scenes or objects, for example.

6. Performance-Based Assessment Eligible Item Types for Performance-Based Assessment (PBA): • Evidence-Based Selected Response (EBSR) • Technology-Enhanced Constructed Response (TECR) • Prose-Constructed Response (PCR) Pg 13 Item Guidelines

7. Part A: What does the word “regal” mean as it is used in the passage? generous threatening kingly* uninterested Part B: Which of the phrases from the passage best helps the reader understand the meaning of “regal?” “wagging their tails as they awoke” “the wolves, who were shy” ‘their sounds and movements expressed goodwill” “with his head high and his chest out”* Sample Evidence-Based Selected Response (EBSR) Item Note: all items included in this presentation are for illustrative training purposes only. They are not representative of PARCC assessment items.

8. Part A : Below are three claims that one could make based on the article “Earhart’s Final Resting Place Believed Found.” Highlight the claim that is supported by the most relevant and sufficient evidence within “Earhart’s Final Resting Place Believed Found.” Part B : Click on two facts within the article that best provide evidence to support the claim selected in Part A. Sample Technology-Enhanced Constructed Response (TECR) Item Note: all items included in this presentation are for illustrative training purposes only. They are not representative of PARCC assessment items.

9. Sample Prose-Constructed Response (PCR) Item Use what you have learned from reading “Daedulus and Icarus,” by Ovid and “To a Friend Whose Work Has Come to Triumph,” by Anne Sexton to write an essay that analyzes how Icarus’s experience of flying is portrayed differently in the two texts. As a starting point, you may want to consider what is emphasized, absent, or different in the two texts, but feel free to develop your own focus for analysis. Develop your essay by providing textual evidence from both texts. Be sure to follow the conventions of standard English. Note: all items included in this presentation are for illustrative training purposes only. They are not representative of PARCC assessment items.

10. End-of-Year Assessment The End of Year Assessment will be: • Focused on supporting Reading Comprehension Claims • Machine scored Eligible Item Types for End of Year Assessment (EOY): • Evidence-Based Selected Response (EBSR) • Technology-Enhanced Constructed Response (TECR)

11. Item Review Committee Charge • Your role is to provide expert CONTENT review of items and tasks. • You will use information provided in this item review training to review the items. • You should focus exclusively on PARCC’s item review process.

12. Item Review Committee Please also note the following: • Passage review committees have already approved the passages according to PARCC content guidelines. • Bias/sensitivity item review committees will apply bias/sensitivity guidelines to all items. • Concerns beyond the scope of your charge as an item reviewer will be placed in the “parking lot” for consideration by PARCC leadership.

13. Item Review Criteria by Core Leadership Group • Does the item allow for the student to demonstrate the intended evidence statement(s) and to demonstrate the standard(s) to be measured? • Is the wording of the item clear, concise, and appropriate for the intended grade level? • Does the item provide sufficient information and direction for the student to respond completely? • Is the item free from internal clueing and miscues? • Do the graphics and stimuli included as part of the item accurately and appropriately represent the applicable content knowledge? (continued)

14. Item Review Criteria by Core Leadership Group • Are any graphics included as part of the item clear and appropriate for the intended grade level? • If the item has a technology-based stimulus or requires a technology-based response, is the technology design effective and grade appropriate? • Is the scoring guide/rubric clear, correct, and aligned with the expectations for performance that are expressed in the item or task? • If the item is part of a PBA task, does it contribute to the focus and coherence of the task model?

15. Previous Steps in the Item Development Process A large group of experts in the area of English Language Arts have previously viewed these items, including: • English Language Arts teachers • ELA assessment experts • English higher education faculty • State Department of Education personnel • School and district administrators The items have been previously reviewed and edited for content area accuracy, alignment to the standards/evidence statements, and grade level appropriateness. This ensures that the items you will be viewing have been vetted.

16. Item Review Process and Criteria for State Educators Step 1: Each reviewer on your team will read the passage/text and the items independently. Step 2: For each item, check to make sure the item meets the evidence statements/standards . If the item can be edited to align with the evidence statements/standards, note in comments how to align the item. If not, reject the item and move on. Step 3: Determine if there are any fatal flaws in the item (i.e. anything that makes the item unusable as written). Document fatal flaw. Step 4: Advise to accept, accept with edits, or reject the item.

17. Fatal Flaws • Use of language in the item in a way that could have unintended consequences • Language that is convoluted and obscure • Note that the flaw should be such that when one raises the issue, the whole review group immediately says, “Oh yes—this can’t go forward as is.”  • If some see “the flaw” and others don’t see it as a flaw, then it shouldn’t be seen as a fatal flaw.

18. Questions?