NWEA MAP Measures, Metrics and Cut-Points for Alternative Education Campuses

1. NWEA MAP Measures, Metrics and Cut-Points for Alternative Education Campuses Colorado Department of Education July 10, 2012

2. Agenda • Purpose • Selection of Measures Process & Considerations • Metrics and Cut-Points for NWEA MAP Achievement Results • Metrics and Cut-Points for NWEA MAP Growth Results • Calculating NWEA MAP Growth Results • Timeline

3. Purpose • Explain methodologies for calculating and reporting NWEA MAP assessment results for accountability purposes on the AEC SPF report (metrics and cut-points). • Answer questions related to NWEA MAP assessment results for accountability purposes on the AEC SPF report.

4. Selection of Measures Process • AECs submitted the Selection of Measures Form July 2. • http://www.cde.state.co.us/Accountability/Downloads/AECSelectionOfAccountabilityMeasures.xlsx • There are a set of state required AEC measures: • Achievement: CSAP/TCAP % proficient/advanced • Growth: CSAP/TCAP Median Growth Percentiles • Postsecondary: Completion rate, Dropout rate, ACT composite • Engagement: Average daily attendance, Truancy rate • Districts may select additional measures, subject to CDE approval.

5. Considerations for Measures • The three most recent years of data will be included. • AECs will receive indicator ratings similar to the traditional SPF: • Does not meet, approaching, meets, or exceeds AEC norms • Each indicator/sub-indicator: • Uses adjusted AEC cut-points relative to each other • Typically uses either normative cuts of 40th/60th/90th percentile or head counts of 40%/60%/90% • Where there are common measures across districts, common metrics and cut-points will be used.

6. Considerations for Overall Results • Based on the indicator ratings, AECs receive an overall point total and plan type. • AECs engage in the same UIP process as all other schools, and have the same consequences for performance. • Note: There is a modified UIP template for AECs to reflect the modified performance indicators.

7. Metrics and Cut-Points for MAP Academic Achievement (Status) Results Of students continuously enrolled for at least 8 weeks

8. Metrics and Cut-Points for NWEA MAP Academic Growth Results % of students achieving target growth from one test administration to the next

9. Calculating NWEA MAP Growth Results • The following slides have been adapted from documents written by Dr. Jody Ernst: • NWEA Use Guidance (07.05.12) • NWEA AEC Research Findings to Inform How to Best Articulate the Assessments Use for Purposes of AEC Accountability • http://www.cde.state.co.us/cdedocs/OPP/NWEAAECWriteUp.pdf

10. Calculating NWEA MAP Growth Results • The results and norms from these studies are all based on Colorado AEC MAP data from 2008-2010 • Insufficient data were available to calculate Colorado AEC growth norms for Science, but Reading, Mathematics and Language Usage norms have been published

11. NWEA MAP Characteristics • Grades K-11 (or students with grade level equivalents of K-11) • Reading, Mathematics, Language Usage, Science* • Fall, Winter, Spring assessment windows • Student-level results are RIT scores * As noted previously, Science growth targets are not available

12. Pulling NWEA MAP Data • Export NWEA data for all subjects in which most students in a grade take the assessment • Include 3 years of data or as many years as are available if your school only recently started giving NWEA MAP

13. Formatting NWEA MAP Data • For each student, compute the difference, in weeks, between each testing occasion (i.e., fall to winter, winter to spring, and/or fall to spring) • Only results based on assessments that were at least eight weeks apart may be used.

14. Calculating each Student’s Pre-test Grade Equivalent • Using the Fall/Winter/Spring RIT Ranges and Fall/Winter/Spring RIT Grade Equivalents, compute the grade level equivalent for each student’s pre-test RIT score • The following tables show the RIT ranges and corresponding grade equivalents for the Fall MAP administration

15. For example, if a student receives a RIT score of 219, they have a Reading level equivalent to that of an 8th grader (regardless of their assigned grade). This value is used as their pre-test grade equivalent.

18. Calculating RIT Growth • Calculate the observed amount of pre-test to post-test growth for each student according to the following: • Fall to Winter = Winter RIT – Fall RIT • Winter to Spring = Spring RIT – Winter RIT • Fall to Spring = Spring RIT – Fall RIT

19. Calculating Target RIT Growth using Seasonal Assessment Windows • If most students are consistently enrolled in your AEC and take the MAP assessment at seasonal intervals, the easiest option for finding a student’s growth target is to use the tables previously published by Jody Ernst. • For each student, use the assessments that give the longest window between pre- and post-test administrations.

20. Calculating Target RIT Growth using Seasonal Assessment Windows • Using a student’s pre-test grade equivalent and the duration between pre- and post-test administrations, determine the appropriate growth target for that student • The following tables show the RIT growth targets for each combination of starting grade equivalent and test window.

21. For example, the student at an 8th grade equivalent for Fall (RIT score of 219), is expected to increase their Spring RIT score by 3 points (to a RIT of 222). For AEC reporting purposes, 3 RIT points is this student’s target growth.

24. Calculating Target RIT Growth using 8-week Enrollment Windows • If students enroll in your AEC throughout the year and for radically differing lengths of time and take the MAP assessment as a pre- and post-test at irregular intervals, consider using growth targets based on 8-week enrollment windows. • For each student, use the assessments that give the longest window between pre- and post-test administrations.

25. Calculating Target RIT Growth using 8-week Enrollment Windows • CDE is currently working to calculate growth targets for by pre-test grade equivalent for 8 week intervals. • These results will be available within the next week or two and will enable AEC’s opting into NWEA and the 8-week interval method to complete their Selection of Measures forms.

26. Determine if Target Growth Achieved • Determine whether each student met their growth target by comparing the actual growth obtained to their target growth. • If a student’s observed growth was greater than or equal to their target growth, the student MET their target. • If a student’s observed growth was less than their target growth, the student DID NOT MEET their target.

27. Aggregating at the AEC Level • Compute the percentage of students meeting their target growth • Number of students that met their target growth (being sure to use only one growth result per student) • Total number of students with pre- and post-test records (using one record per student)

28. Completing Selection of Measures Form • Fill in your AEC’s aggregated percentage of students meeting their target growth • Fill in your AEC’s total number of students with pre- and post-test records (denominator)

29. Assigning a Growth Rating • CDE will assign a rating to your AEC’s reported NWEA MAP growth • This calculation is done separately for each of the three subject areas by comparing the percentage of students meeting their target growth against the following state cut-points:

30. Process/Timeline July 25: NWEA Results to CDE

31. Resources • Alyssa Pearson, Executive Director • pearson_a@cde.state.co.us , 303-866-6855 • Marie Huchton • huchton_m@cde.state.co.us , 303-866-6203 • Donna Morganstern • morganstern_d@cde.state.co.us , 303-866-6209 • Somoh Supharukchinda • supharukchinda_s@cde.state.co.us , 303-866-6778 • http://www.cde.state.co.us/Accountability/StateAccountabilityAECs.asp