NEW JERSEY ALTERNATE PROFICIENCY ASSESSMENT (APA): STANDARD SETTING

1. NEW JERSEY ALTERNATE PROFICIENCY ASSESSMENT (APA): STANDARD SETTING Willa Spicer, Deputy Commissioner Barbara Gantwerk, Assistant Commissioner, Division of Student Services Timothy Peters, Director, Office of State Assessments State Board of Education, July 15, 2009 New Jersey Department of Education

2. Overview of the Alternate Proficiency Assessment (APA) • Federal law requires that students with disabilities be assessed in the same grade levels and content areas as administered in a state’s general assessments; • The APA measures the progress of students with severe cognitive disabilities; • It must measure academic skills, not functional or motor skills; • It is a portfolio assessment, using evidence developed by teachers over a several month period;

3. Overview of APA, cont. • Students assessed through the APA are not able to access the general assessments and are not being instructed in the Core Curriculum Content Standards in the same depth or breadth as students who take the NJ ASK or HSPA; • Previous reviews by the USDOE found the NJ APA insufficiently grounded in grade level standards, so we have redesigned the program in consultation with the USDOE; • NJ’s APA design provides grade level “links” to specific cumulative progress indicators (CPIs) in the CCCS by which teachers can measure student achievement on the APA.

4. Overview of APA, cont. • Approximately 8200 students are assessed each year, from grades 3 through 8, and grade 11; • Student portfolios are scored April-May each year by trained scorers; • Portfolios are scored for “independence,” “complexity,” and “performance,” with performance given extra weight.

5. APA Standard Setting • Held June 9-12, 2009, following scoring; • 86 NJ teachers participated, including special education and general education teachers from across the state, demographically representative; • Standard setting panelists are not scoring the portfolios; they are setting performance standards; • In the standard setting process, expert judges – teachers – engage with actual student portfolios and with their colleagues in discussion, over several days, facilitated by measurement specialists; • Proficiency Levels: Partially Proficient; Proficient; Advanced Proficient; based on raw score points achieved.

6. Validation • Teachers make recommendations for the cut scores and discuss impact data in several rounds, culminating in final set of recommended cut scores. • On final day, an articulation review among a sub-group of teachers from each panel considers the group’s recommendations as a whole; • The APA panel’s finally recommendations are reviewed by the Office of Special Education staff and then by senior staff; • Some adjustments were made to assure articulation across grade levels; • See following slides for the department’s recommended cut scores and impact data.

7. Summary of Recommended APA Cut Scores and Impact, with Comparisons to 2008: Language Arts Literacy (LAL)

8. Summary of Recommended APA Cut Scores and Impact, with Comparisons to 2008: Mathematics and Science

9. Next Steps • Release of APA score reports to districts • Ongoing support and assistance for teachers engaged in APA process • Questions?