Information on New Regents Examinations for 2007-08 SCDN Presentation September 19, 2007

1. Information on New Regents Examinations for 2007-08SCDN Presentation September 19, 2007 Steven Katz, Director Candace Shyer, Bureau Chief Office of Standards, Assessment and Reporting

2. What is new in June 2008? • Physical Setting/Earth Science Performance Test (Part D) • Integrated Algebra Regents Examination

3. Physical Setting/Earth Science Performance Test (Part D) • Format of new Performance Test is very much like the old • Part D: hands-on portion of the Earth Science Regents Exam given in class two weeks before the written exam • 22 points (old version 23 pts.) which will be scaled to 15 points • 4 stations (old version 6 stations)

4. Part D (con’t) • 9 minutes per station for a total of 36 minutes (old version same total time but 6 minutes per station), plus time to move between stations • Core covered • rocks/minerals • mass/density • epicenters (new) • elliptical orbits (new)

5. Part D Training • New materials • Administration Booklet, Station Directions for Students, Rating Guide and Student Answer Booklet • Training workshops in Fall 2007 • Train-the-trainer workshops in seven locations • The trainer teachers will help train the rest of the teachers in districts on how to administer and rate the new performance test

6. Part D (con’t.) • Standard Setting – end of September 2007 • 20-25 teachers will come to Malta for two days • Make recommendations to SED on a “cut score” or a minimum score for proficiency

7. Integrated Algebra First date administered: June 17th First step in the transition from Mathematics A and Mathematics B to Integrated Algebra, Geometry, and Algebra 2/Trigonometry

8. Mathematics Regents ExaminationsImplementation / Transition Timeline Approved by the Board of Regents 12/8/05 .

9. In late October 2007, Integrated Algebra Regents Examination Test Sampler will be posted on SED web site. • examples of the types of questions, the formatting, and the scoring guides • may be duplicated for classroom instruction • will be found at: www.emsc.nysed.gov/osa/mathre • Graphing calculators are required for this examination

10. Integrated Algebra: Test Specifications • general picture of the structure of the exam • test specification meeting in 11/06 • teachers recommendthe number and type of questions to be tested and • what percentages of the total credits on the test should align with each content strand • posted at: www.emsc.nysed.gov/osa/mathre

11. The Role of Riverside Publishing Company • Division of Houghton Mifflin • Vendor contracted to develop the test in conjunction with the SED and NYS teachers. • Regents Examination items are written by NYS teachers!

12. Item Writing Workshop • This is the first step in creating a bank of items to be used on field tests and operational tests. • For new math Regents Exams, Riverside Publishing runs one or more 4 day item writing sessions each year. • Teachers receive training on writing items and scoring rubrics from Riverside Publishing and SED staff.

13. Item Writing Workshop (con’t.) • New York State teachers write multiple choice and constructed response questions and their scoring rubrics. • Items are sent to SED for review by staff and consultants. • Items are then further reviewed by teacher review committees.

14. Item Review workshop (con’t.) • Second step in preparing items for field tests and operational tests. • Riverside Publishing runs one or more item review sessions. • Teachers receive training on reviewing items and rubrics written by item writers. • Teachers review items and rubrics to make sure they align with the assigned content performance indicator.

15. The Field-Test Phase • Purpose: Our first round of field testing helps to identify and select the best questions for the final operational test forms and to identify bias in a given test question

16. The Test Development Process:Field Testing (con’t.) • Items developed at the item writing workshop are used to assemble field test forms. • Several committees of teachers work with Riverside/SED staff to prepare and review a series of field tests. • All possible field test items under go additional content review prior to actual field testing • A field test sample is selected for all questions and students take those field tests. These are SECURE test forms and schools must maintain their security. Schools may not make any notes or photocopies of the field tests.

17. The Test Development Process:Field Testing (con’t.) • The field tests are returned to SED, checked in, rubrics are refined, and the papers are scored. • Field tests undergo further statistical analysis that looks at how the items perform individually as well as how they perform together as part of a test form using both classical and Item Response Theory statistical analysis.

18. The Importance of Field Testing • Essential step in developing valid and reliable tests for all students in the State. • Ensures that questions are tied to the Standard and the Core Curriculum and are fair and representative for all students. • Provide experience for students in responding to the types of questions they will be exposed to on the Regents Examination.

19. Importance of Field Testing • Teachers throughout NYS play a vital role in making this process work.

20. The Test Development Process: Operational Phase • Based on field test results the operational exams are assembled and reviewed. • A rating guide is prepared (by teachers working with Riverside and SED staff) using actual student work from field testing. • An exam is presented to a “Final Eyes” review panel of 3-5 teachers from around the State.

21. The Test Development Process: Operational Phase (con’t.) • Students take the examination. • For June 2008, schools will be required to ship their scored answer sheets to contractor (Pearson Educational Measurement) by the end of the day following the day of administration of the examination for score and data collection and analysis. • The exam, answer sheets and scoring materials are printed and shipped to schools.

22. The Test Development Process: Operational Phase (con’t.) • conversion chart for determining each student’s final examination score from the total raw score will be posted on the Department’s web site no later than the Rating Day, Thursday, June 26.

23. Standard Setting • SED will use the same processes that were used during standard setting for the Grades 3–8 ELA and Mathematics Tests. These include: • Conducting the standard setting after the examination has been administered using operational testing data rather than field testing data; • Implementing the Measurement Policy Review Forum; • Standard setting using the Item Mapping Method with New York State general education and special population teachers; • Reviewing final results with the Department’s Technical Advisory Group prior to submission to the Commissioner.

24. Further questions • E-mail: emscreminfo@mail.nysed.gov • Providing feedback on State examinations through the Teacher Evaluation Form: http://www.emsc.nysed.gov/osa/exameval