Tools for Schools: Developing End-of-Course Assessments Statewide

1. Scott Cook, ISAT Coordinator Rob Sauer, Deputy Superintendent of Innovation and Choice Carissa Miller, PhD, Deputy Superintendent of Assessment Tools for Schools: Developing End-of-Course Assessments Statewide

2. A. Purpose, goals and uses of End-of-Course Assessments (EOCs) B. EOC test development plan C. Mentor/mentee model of development D. Other tools available for districts E. Creation logistics, getting involved F. Questions and comments Outline

3. Variety of uses Classroom tests/quizzes Interim/benchmark assessments Summative assessments EOCs in classrooms (15% of grade) Mastery tests Graduation exams Assessment Purposes

4. Alignment to standards Validity Are we testing what we intend to measure? Reliability Can the measure be repeated with confidence? Pearson correlation, Cronbach’s alpha Differential Item Functioning (DIF) – performance among subgroups Rigor Sufficient rigor for the course/purpose? Comparability Scoring and Reporting *American Educational Research Association, (1999) “Standards for Educational and Psychological Testing,” Washington, DC: AERA. Good Testing Practices*

5. Used as a tool to inform instruction Potentially for alignment of curriculum Textbooks and course outlines vary Augmenting assessments within district/schools Item banks State EOC Uses

6. Provide Sample End-of-Course examinations in: Biology, Physical Science and Earth Science Geometry, Algebra 1, Pre-Algebra Provide an Open source Item bank in these course areas- For local assessment For formative assessment to take place EOC Test Development Plan

7. Test Development: ISAT/EOC Comparison ISAT Start to finish: roughly a 2-year process Trained item writers follow standards, content limits; depth and range of knowledge, balance of representation Items are reviewed by Idaho teachers before piloting Piloted items are reviewed again for statistical analysis and alignment to standards and content limits

8. ISAT Test Development [cont.] Bias review Item correlation DIF analysis Distractor analysis % correct values Building test forms Forms review Census test ready

9. EOC Test Development Plan Multiple choice format Combine item writing and training with mentor/mentee model Using experienced EOC item writers and inviting teachers from schools in “Needs Improvement Status”--access SI dollars Professional development a positive by-product of process Building item bank with existing items from Boise, Meridian, and Twin Falls [thanks!]

10. Tasks ahead…………. SDE must finalize or write standards which inform sample Test Blueprints- Geometry and Algebra course outlines exist Create individual online item cards- @ 1,800 for 6 courses, load in SchoolNet Create list of focus words that inform search function

11. Experienced item writers are paired with novice items writers 10 individuals per course group Whole group trains in basics of item writing Groups begin to code items to test blueprints EOC Workshop, Nov. 2010

12. Groups select existing items that have worked well in districts for sample test forms Groups write new items to flesh out test forms Test forms are created Test forms are reviewed Items not used in forms revert to bank Workshop Week [cont]

13. SchoolNet will house test forms and item bank Educators will access through Single sign in portal and will need permission to use EOC application Ability to create formative assessment, local EOCs, unit tests by searching standards or focus words--- ex., “ Distributive property.” Housing of EOCs

14. SCHOOLNET Demo Chapter 4: Creating Tests The Create page is where you create new tests and new test items. The test creation process likely began with the test blueprints and high-level benchmarking goals being set prior to the actual test creation process in Assess. Creating the test using the form below creates a test shell where items are then added from Item Central, or are authored right from the test details page. After the test content is entered, Assess supports a common workflow that takes the test into various stages, including a Public Draft that allows content and assessment specialists to review the items, alignments, metadata, and online testing display collaboratively. To generate a test, see: Step 1: Getting Started Step 2: Choosing an Item Type Step 3: Working within Item Types See also: Test Stages Test Detail Customizing Tests Modifying Tests Deleting Tests Creating Passages: Getting Started Creating Passages: Adding Content

15. Course Outlines Courses of Study Sample Lesson Plans linked to Standards Formative Assessments Identify and Share Links to Sites and Best Practices Other Tools Available

16. When? - November Where? - Boise How will participants be invited? - Federal Requirements - We will be contacting districts that are eligible - Brief application will be required - Mentors will be selected All districts will benefit Creation Logistics

17. Teachers access items and test forms on SchoolNet platform Local access, local control Finalizing course test forms, QA items, training and rollout Winter 2011 Final thoughts

18. Questions?

19. If your district is interested in participating: Rob Sauer rcsauer@sde.idaho.gov (208) 332-6934 Contact Us