Assessment for Learning: OARS/INSPECT Update

1. Assessment for Learning: OARS/INSPECT Update Fresno Unified School District May 12, 2008 Research, Evaluation and Assessment

2. Assessment Update • Today We Will: • “Frame” Assessment for Learning into our Instructional System • Discuss the process for using the Inspect item bank and the OARS processing software • Provide initial user names and passwords for the early implementers at your school • Discuss site-based scanning options

3. Fresno Unified Assessment Philosophy 3 Assessment Perspectives: • Assessment for Accountability • (State-mandated) • Assessment of Learning • (periodic status checks linked to standards) • Assessment for Learning • (assessment during learning)

4. Aligned Instructional System Taught Curriculum The Learned Curriculum Assessed Curriculum Written Curriculum

5. Assessment for Learning– Why? • Students understand learning intentions/objectives • Students understand how good is “good enough?” • Analysis of student responses allows the teacher to identify student understandings and gaps in mid-stream • Teachers study how to focus the “next” instructional lesson • Students begin to ‘own’ their own learning

6. Assessment for Learning– What? • Single Standard AFS Quizzes in AiS • Performance Task AFS Assessments in AiS • “Build an Inspect Quiz” in OARS

7. When: “Build an Inspect Quiz” in OARS • “I want to know what my students can do before I start a unit or lesson” • … in order to determine and shape the instructional unit around student needs. • “I’ve spent 3 days on this skill/concept, and I want my students and me to be aware of which components of the skill still need work” • … in order to differentiate and focus the remaining instructional time. • “My students now have the groundwork for this skill/concept, but do they “get it” at the level of depth required on an assessment?” • … supporting a rigorous interpretation of the standard/skill • “I’ve completed instruction for this standard. What have students mastered?” • … in order to determine what students know, and who needs additional support. • “Students have practiced this standard as part of an “extra support” intervention, how can I assess their knowledge?” • …In order to follow up on students needing extra help. • “I want to use an assessment item in the middle of my lesson” • … to check for understanding

8. What: “Build an Inspect Quiz” in OARS • Navigate to http://www.oarsaccess.net/ • Select Fresno in the “Select your District” window • Log in. Initial log ins are… • Username: firstname.lastname, using the Sharepoint format • 1st Password: birth month and last 2 digits of birth year… September 1963 would be ‘0963’ • Usernames and passwords also handed out today • This is NOT AiS!!!! The username and password for OARS is different than in AiS • Three “Download Resources” may be helpful for you: “OARS in a Nutshell,” “OARS Users Manual,” and “ScanOARS Download” (when you are ready for scanning)

9. Go to www.oarsaccess.net, choose Fresno Unified and log-in

10. What: “Build an Inspect Quiz” in OARS • To Build a Quiz …. • Select the “INSPECT” tab at the top of the main menu • Select the “Create a new test” button • Complete the “Test Information” boxes • Select the Course, Strand and Standard Information • Select items for the quiz by choosing the “ADD” button for each item you select. You can see the directions and the distractor rationales for an item by clicking the boxes below each item. • Note: for ELA items, there is an option to look at other items that use the same reading passage. If you are trying to create a “one passage” quiz, you might select this to identify other items associated with the passage. • An Inspect quiz should be short… 5-6 items only

11. “Inspect” tab to build a quiz

12. The “Inspect“ Main Screen Create a new Quiz here

13. Quiz Description Select Items View Alternative versions of an item Additional Items to view

14. What: “Build an Inspect Quiz” in OARS • To Build a Quiz (Continued) …. • Review the draft of your quiz • Save the quiz… after you click on save, you can still edit/delete, until you ….. • … Print and Lock (when you do this, a print copy will appear in a new window). • You can also choose to share the quiz, or keep it just for your class • Print the quiz, and you have several print options available for how you’d like the quiz to look… • Bubbles or letters for the answer choices • Answers highlighted or not • Answer key accompanying • Print a copy with distractor rationales

15. Review, Print drafts, and Print-Lock Quizzes here Print Answer Sheets for Locked Quizzes

16. What: “Build an Inspect Quiz” in OARS • Additional Information about building quizzes…. • Do not quarantine an item (this locks others out from using it) • Some items are quarantined at the system level for inclusion on a District-wide test (CAHSEE Diagnostic tests) • Some questions have an extra version created (e.g., with just a small change to an answer choice). The “View Alternate Item” button allows you to see all versions of an item, and pick the one you want to use.

17. What: “Build an Inspect Quiz” in OARS • To Print Answer Sheets …. • Teachers with multiple grade levels, or teaching multiple classes (secondary) will need to select a “section” of their classes for which to print answer sheets. Click on the “Sections” tab on the blue strip at the top of the screen, then • Click on the program menu, and select “Inspect Item Bank” • Click on your name • If you are selecting a subset of all your students to take the test, select them here, otherwise the whole student list will be uploaded • From the “Inspect” tab, select “Print” answer sheets and follow the directions there. • Have students take the quiz!!

18. For teachers with multiple grade levels or classes, select your class(es) here Select Inspect Item Bank

19. What: “Build an Inspect Quiz” in OARS • After you give the Quiz … • For the next few days …. Quizzes must be hand-scored, just as with a classroom-developed assessment, unless your school is already using ScanOARS. • Within a few days, a manual data-entry screen will be available. Though not ideal, if you keep the quiz short, the data entry will not take long. Go to the “Enter Scores” tab, and select your assessment. You should be able to enter correct/incorrect, or enter in the students’ actual answer choices, to get a response analysis. • View the various reports available in the “Reports” tab

20. Notes on Scanning Answer Documents • Many elementary schools are already using a Brother Scanner to scan and upload answer documents. The Brother works fairly well, but is not supported by the District if it isn’t working! • FUSD is working with OARS to develop the functionality of an HP scanner, which will be supported if it isn’t working– but more development work needs to be completed before we recommend it. • Long Term: OARS will work with a Xerox machine that has scanning capabilities. FUSD is looking into creating upgrades with Xerox for a high speed solution • Ideal: OARS works in an on-line format. If students have access to computers, this may be a great option as well.

21. OARS-Inspect for Now … • … is ready to explore, begin creating tests, and using during/after learning to inform next steps. • … early adopters can race around in OARS to see what they can do. • … can be used, with manual data entry within a few days (or with an already functioning scanner, for Reading 1st schools) • … if you don’t have a scanner already functioning, wait until we can make a good decision about which solution will work best. We will be in contact with you– purchasing a scanner will not be your responsibility.

22. OARS-Inspect for Next Year … • FUSD will be working with instructional coaches to become familiar enough with OARS/Inspect to be the first resource for support. • Support for HP scanner troubleshooting will be available • The Xerox option will be fully explored • FUSD will continue professional learning about using Inspect in the Aligned Instructional System as assessment for learning.

23. Cycle of Teaching Planning of Instruction Planning of Instruction Reflection Instructional Strategies Reflection Assessment Analysis Analysis

24. Planning: What will I do differently to ensure that every student learns this std. (strategies, resources, etc.) Goal: Create a plan to re- address this std. including progress monitoring Resources: Curriculum TE, Professional Sharing Planning of Instruction Reflection: What/How did I teach? What were student misperceptions? Why? Goal: Determine why students may not have learned stds. to proficiency Resources: Response Analysis Tools (if Applicable), Curriculum TEs Implement: Implement planned instructional strategies for agreed upon timeframe before re- assessing to determine proficiency Reflection Instructional Strategies Cycle of Teaching Analysis: How successful was our teaching? What did our students get/not get? Goal: Identify at least one common standard of focus for re-teaching/pre-teaching Resources: Assmt. results, CST Blueprints, Curriculum Map Assessment: Did they get it now? Goal: Administer common “assessment” (district or teacher created – e.g., AFS/INSPECT teacher created quiz, student work). Analyze at pre-determined time with grade level Assessment Analysis

25. School Administrator Do’s and Don’ts Do … • Share initial usernames and passwords with teachers • Encourage teachers to “play around” to create a quiz • Encourage teachers to think about how they could create a short quiz for an “essential learning” at the end of the year • Ask your instructional coaches to log in and get familiar with the system • Share with teachers that this doesn’t replace instruction, or become a worksheet Don’t … • Turn assessment for learning into an accountability measure by analyzing Inspect results from classrooms to evaluate teachers • Buy a scanner yet, if you don’t have one now!