Grade 11 NECAP Report

1. Grade 11 NECAP Report Rhode Island Department of Education February 27, 2008

2. First Release of NECAP Grade 11 For the first time, performance results from the New England Common Assessment Program (NECAP) for Grade 11 students are being released today by Governor Carcieri and Commissioner McWalters. Through a collaborative partnership believed to be the first of its kind in the nation, public school students from New Hampshire, Vermont, and Rhode Island participate in common assessments to measure proficiency in reading, mathematics and writing. Students in grades 3-8 have participated in the NECAP assessment program for the last 3 years. Those results were released to the public on January 31.

3. Rhode Island Data 61% proficient or better in reading at Grade 11

4. Rhode Island Data 37% proficient or better in writing at Grade 11

5. Rhode Island Data 22% proficient or better in mathematics at Grade 11

6. When we look at the three subject areas… • Reading and Writing grade 11 achievement results mirror grade 8 in the first year of NECAP • Mathematics results do not. Why? • What questions should we be asking? • Was it the test? • Is the test too hard? too long? too different? How did we get here?

7. First defined what students should know and be able to do - Grade Span Expectations (GSEs) Teachers developed the GSEs based on national standards and best research Developed test specifications (items, length, etc..) Developed and reviewed test items Teacher committees reviewed every item for content and bias andthen …… The NECAP Test Development Process

8. The NECAP Standard Setting Process • Gathered additional student data - • Teacher judgments; NHEIAP and NAEP historical data • Considered the review of our test and our standards by national content experts and professional organizations • Set Achievement Levels: • Teachers in each subject area from the 3 NECAP states: -Took the test -Wrote Achievement Level Descriptors -Developed Item Maps -Recommended cut scores for each achievement level

9. What did the 2007 Reading test look like? • After responding to a small number of vocabulary questions, students were asked to read 3 long and 3 short literary and informational passages and answer questions after each. • There were 42 multiple choice (1 pt.) questions and 9 constructed response (4 pt.) questions. • The test was given in 2 sessions, each designed to take about an hour. Test administrators were instructed to ensure that 90 minutes be allowed for each session.

10. What did the 2007 Writing test look like? • The high school writing test was a single item test. This extended writing exercise required students to respond to informational text. All students were scored on the same task. • Each student also responded to a second test item that was not reported as part of their individual student score. This second itemsampled one of the other forms of writing in the high school GSEs (persuasive, response to literary text, report or procedural writing). All forms of writing were sampled within each school. Results from these sampled items are reported at the school, district and state levels to provide program-wide feedback.

11. There were 64 points on the test More than a third of the test points (24) came from multiple-choice items. The remaining points (40) came from 1, 2 and 4 point constructed response items. The test was given in 2 sessions, each designed to take about an hour. Test administrators were instructed to ensure that 90 minutes be allowed for each session. What did the 2007Mathematics test look like?

12. What did 9,157 RI students say about Reading? When asked how often they read… 40.1% said they read a few times a week or almost every day in their free time. 37% said they read a few times a month or almost never during their free time. How do they define “reading”? 12

13. What did 9,157 RI students say about Reading? …cont’d. When asked how they get information about things that interest them… • 15% said …they use hard-copy printed materials to find information that interests them. • 47% said …they usually use a computer to find information about things that interest them.

14. Maybe they’re reading more than they (and we) think they are. Are we teaching 21st century reading skills?

15. What did 9,157 RI students sayabout Writing? How difficult was the writing test? 45% said the test was about the same as regular work. 20% said it was harder than their writing school work. 14% of students said it was easier. How hard did you try on the writing test? 46% said about the same as I do on my regular work. 20% said they tried harder on this test. 13% said they did not try as hard as they do on their regular work. 15

16. What did 9,157 RI students sayabout Writing? 16

17. What did 9,157 RI students say about the Mathematics test? • 42% said the test was more difficult than typical tests in school • Only 39% said they have to explain how they solved a math problem almost every day in school (like on the test) • 55% said they used a calculator on the test when allowed

18. What else did RI students say about the Mathematics Test? • 39% said they had enough time to finish all test items and to check their work. • 19% said they had time to finish, but not check their work • 13% said they felt rushed, but did finish. • 10% said they did not have enough time.

19. What did Rhode Island Test Administrators say about the Mathematics Test? • 84% of respondents said that most students gave a good effort on the test • 29% said students finished session 1 of the test in under an hour (the non-calculator session). • 28% said students finished session 2 of the test in under an hour • 77% of 631 respondents said to shorten the time per session or keep the timing as is

21. What did Rhode Island Test Coordinators say about the GSEs? • Only 44% of respondents said the GSEs have been fully integrated into their curriculum • 37% said they have just begunor are working on implementation and integration of GSEs • 19% said they have not yet started to integrate the GSEs into curriculum.

22. Released Reading Test Item –Constructed Response (4 pts.) By Naomi Shihab Nye

23. Reading Test Item – Scoring Guide

24. Extended Writing Prompt(12 pts.) Each student response was scored by two independent scorers using a 6 pt. rubric. (2 x 6 = 12 maximum points)

25. Scoring Rubric for the Writing Prompt

26. The manager of a music store ordered 20 new violins. She ordered some of two different models- the standard and the deluxe. Each standard violin costs $500, and each deluxe violin costs $800. If the manager spent exactly $11,500 on these violins, how many deluxe violins did she order? Show your work or explain how you know. Released Mathematics Test Item – Short Answer(calculator allowed)

28. Released Mathematics Test Item – performance information • This item was worth 2 points • The p value for this item is .33 • 12.6% of students left the item blank • 48.9% of students attempted but earned no points • 11.1% of students earned 1 point • 27.3% of students earned 2 points

29. 4 point Constructed Response (CR) Item

31. Of 20 random students at proficiency (1140) : 5 (25%) scored 4 pts 0 scored 3 pts 9 (45%) scored 2 pts 1 (5%) earned 1 pt 4 (20%) earned 0 pts 1 left it blank Of 20 random students at the median (1135): 1 (5%) scored 4 pts 2 (10%) scored 3 pts 3 (15%) scored 2 pts 3 (15%) scored 1 pt 9 (45%) earned 0 pts 2 (10%) left it blank How did students do on this mathematics CR item?

32. Some reflections on RI Standards… • It’s the first time we have assessed the Grade Span Expectations (GSEs) - our current state standards. • The standards are different than the former content standards used for the NSRE • We listened to community and post-secondary feedback, looked at current research, and updated our standards to include what students need to know and be able to do to succeed in the 21st century

33. More reflections on RI Standards… • The Reading standards remain consistent with earlier standards and the 2007 scores reflect that. • Writing results are less than acceptable, but are consistent with feedback received from community employers, colleges and national research. • Students’ writing skills need to improve. To do this, students must write in all content areas every day. • Our students did not meet our mathematics performance expectations. • These mathematics results arealso consistent with feedback received from community employers, colleges and international studies.

34. Some questions to be answered… • What do the students say about the results? • In what areas of the curriculum are students struggling? • Are students reading a variety of texts, including informational texts? • Are instructional materials and practices used in RI high schools aligned to the GSE’s?

35. Some more questions to be answered… • Do all students have access to the full curriculum? • What is the impact of ‘tracking’? • How does instruction address student preconceptions and misconceptions in each content area? • How does instruction create a balance between procedural and conceptual knowledge?

36. What is the RI Department of Education doing to support RI schools, districts, and students? • The mathematics results are a call to action that RIDE will lead over the next 12 months. • Personal Mathematics Plans, similar to RI’s Personal Literacy Plans, must be developed for students who are performing below grade level. • Supports to educators will be expand by requiring prospective middle school teachers to pass content examinations in order to be certified, focusing I-Plans on mathematics when needed, and providing professional development.

37. What is the RI Department of Education doing?….. continued • An expert task force will lead districts through a program alignment to ensure that mathematics curriculum and materials are aligned to the GLE and GSE. • RIDE will work with higher education to review the mathematics preparation within all Teacher Training Programs. • Work continues with the Pre K-16 Council and our 2 and 4 year colleges to define what is necessary for students to be ‘ready on day one’ for work, military, or post secondary education.

38. What is the RI Department of Education doing?….. continued • In addition to RI focus groups , New Hampshire, Vermont and Rhode Island will conduct a three-state conversation with mathematics teachers to review pedagogy and ‘habits of mind’ practiced in our schools. • RI’s Statewide Curriculum include appendices with resources to support the implementation of the GLEs and GSEs. • Each year, NECAP test items, support materials and student work samples are released for use and analysis by teachers and students. • More data will be gathered and analyzed from the results, from teachers, and from students.

39. Final thoughts… • These results offer more questions than answers: let’s use them as a starting point, and see them both as a challenge and an opportunity • Lowering our standards is not an option: we need to make the changes necessary to help our students rise to meet this challenge • We must find out what our schools, students and teachers need to help them meet the standards necessary for living and working in the 21st century. • Mathematics education must change.