1 / 86

Welcome to SBAC 04 Vendor Update Please be patient, we will begin soon.

Welcome to SBAC 04 Vendor Update Please be patient, we will begin soon. Please note that this presentation was originally used to present to SBAC state representatives for the purposes of providing background on SBAC 04 providing overview of documents currently available for review

melchior
Download Presentation

Welcome to SBAC 04 Vendor Update Please be patient, we will begin soon.

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Welcome to SBAC 04 Vendor Update Please be patient, we will begin soon.

  2. Please note that this presentation was originally used to present to SBAC state representatives for the purposes of • providing background on SBAC 04 • providing overview of documents currently available for review • describing needs for feedback. • It has been adjusted for the purposes of providing information to the vendor community about the status of SBAC 04.

  3. SBAC 04 Showcase 1 Vendor Update Friday, January 6, 2012

  4. SBAC 04 Showcase 1, January 4 0 Review Agenda and Logistics 1Welcome + Introductions 2Overview of SBAC 04 3Instructions for Feedback 4Evidence-Centered Design 5 Stimulus Specifications 6 Item Specifications Introduction 7 ELA Item Specifications 8 Mathematics Item Specifications 9 Technology-Enhanced Item Specs 10 Performance Task Specifications 11 Accessibility + Accommodations 12 Style Guide 13 Bias and Sensitivity Guidelines 14 Questions and Answers

  5. Logistics for Today’s Showcase Meeting • Participants’ audio connections have been automatically muted to reduce background noise. • Please use the WebEx chat window to forward your questions to Kevin King. • There is a lot of material to cover in two hours today, so we will be adhering tightly to a timeline.

  6. Welcome + Introduction Kevin King: IDWG co-chair Tony Alpert: COO Christyan Mitchell: PMP 1

  7. Overview of SBAC 04 2

  8. SBAC 04 Project Introduction • Purpose of Item Specifications: Develop item/task/stimulus specifications and guidelines to guide item/task production. • Item/Task Specifications will link CCSS, through the SBAC Content Specifications, to item/task development. • These materials will be used by item/task writers and item/task reviewers. • Collaborative process between SBAC leads and Measured Progress/ETS.

  9. SBAC 04 Project Introduction • Item Specification materials taken in isolation from the SBAC Content Specifications will be challenging. • There are a lot of materials to be reviewed as part of this showcase. • The webinar will highlight overall structure and process.

  10. The Purpose of SBAC 04 Project To work with SBAC to develop Item/Task/Stimulus Specifications and Guidelines: • Component 1: Item Specifications, Style Guide, Stimulus Specifications • Component 2: Technology-Enhanced Item Specifications • Component 3: Performance Task Specifications • Component 4: Bias and Sensitivity Guidelines • Component 5: Accessibility and Accommodations Guidelines

  11. SBAC 04 Functional Group Organization

  12. SBAC 04 Project Players and Timeline • SBAC • Item Development Work Group • Performance Task Work Group • Accessibility and Accommodations Work Group • Measured Progress / ETS Collaborative • Subject Matter Experts • December 1– February 29 • Showcase 1 today • Showcase 2 late January • Showcase 3 mid February

  13. SBAC 04 Project Players and Timeline Draft documents will be presented today: • Mathematics and ELA Stimulus Specifications Topics draft • Mathematics Item Specifications draft materials • ELA Item Specifications draft materials • TEI Specifications draft materials • Performance Task General Specifications draft • Accessibility and Accommodations Guidelines draft materials • Style Guide draft

  14. The Purpose of the SBAC 04 Project • The ultimate purpose of specifications and guidelines is to strengthen the SBAC validity argument in an evidence-centered design. • The Content Specifications reflect a break from conventional testing paradigms. SBAC has established claims that the scores from the summative and interim assessments must be able to support. • The Item specifications will use the claims and assessment targets as stated in the Content Specs to guide the work of item/task specifications.

  15. The Purpose of the SBAC 04 Project • Today’s Showcase is a presentation of the first in a series of iterative drafts. • Rough drafts in different stages of development will be presented. • We are seeking input early to allow for course correction and member input. • We are using two levels of feedback: • Big picture, more general feedback/comments • Use the feedback form(s). • Are we on the right track? Do the format and proposed contents make sense and/or meet the needs of SBAC? • Be as specific as you can. • Specific, detailed feedback/comments • Use PDF mark-\up tools to comment right on the PDF(s). • Be specific.

  16. The Purpose of the SBAC 04 Project • Today is an early view of the plans and drafts for the SBAC 04 body of work. • Feedback received from SBAC states will provide: • Guidance on general approach. • Validation of work done so far. • Suggestions for missing content. • Suggestions for direction change. • Detailed suggestions for revision on content “so far.” • Next Showcase (late January) will provide the opportunity to dive into more detailed drafts. Each member state will be asked to contribute to review of the next draft.

  17. Instructions for Feedback 3

  18. Evidence-Centered Design 4

  19. Evidence-Centered Design How to view Evidence-Centered Design (ECD)? • ECD is as a logical, systematic approach to test design and development based on the principles of evidentiary reasoning. • ECD is not used as a rigid set of procedures. • ECD gives us tools for linking all of the steps of test development to the collection of the evidence required to support the claims to be made about test takers.

  20. Evidence-Centered Design What does ECD do? • ECD does not make the traditional test development steps obsolete. • ECD offers tools to help get the steps done. • ECD helps clarify, document, and link the steps in test development, but the steps still need to be done whether ECD is used or not .

  21. Evidence-Centered Design Steps in ECD • ECD starts with claims about the KSAs of test takers—what SBAC wants to say about students on the basis of assessment results. • SBAC has provided high-level claims for summative tests in ELA and Mathematics.  • ECD requires that evidence be provided to support each claim. • SBAC has provided descriptions of the evidence needed to support the claims. • The SBAC Content Specifications provide the claims and descriptions of evidence needed as the basis of ECD.

  22. Evidence-Centered Design Steps in ECD • Each item/task (SR, CR, PT, TE) has to elicit evidence for one or more claim(s). • Item Specifications will set specifications for the tasks, and rules for their accessibility and fairness. Each task will be linked to evidence for a claim or claims.

  23. Evidence-Centered Design Interconnectedness of ECD Steps • Earlier decisions affect all later phases. • Any later phase could require changes in earlier phases. • Phases provide information for each other, both forward and backward. • Iteration and recycling throughout the phases is expected and helpful.

  24. Evidence-Centered Design Benefits of ECD • ECD strengthens validity argument. • ECD aligns all parts of the test design and development process. • Every step of test development from setting specifications to reporting scores is logically linked to previous and following steps.

  25. Stimulus Specifications 5

  26. Stimulus Specifications • What do we mean by stimuli in assessment? • ELA (e.g., text, web pages, speeches) • Mathematics (e.g., graphs, data displays) • Why provide specifications for stimuli? • How are we determining the specifications? • What kinds of information will be included? • What progress have we made thus far?

  27. Stimulus Specifications • The draft list of topics is available. • What we’ve asked SBAC states for feedback includes: • Are these topics the right topics to include? • Are there any topics you would add? • Are there any topics you would omit? • Are there any resources you would like to recommend to us? • Please add any additional comments you think would be helpful.

  28. Item Specifications Introduction 6

  29. ELA and Mathematics Specifications Specifications will have 3 main sections: • General Specifications • Overview of best practices for writing various types of items • Complexity Guidelines for Items • Grade-appropriateness of various contexts, etc. • Instructions to writers for completing the item forms • Item Specification Tables • General information and guidance related to assessing the grade-level claims and targets • Item Forms • Specificinformation about each sample item/task, including its relationship to the claim(s) and target(s)

  30. *QTI - Question and Test Interoperability Standards

  31. Item Specification Tables

  32. Item Specification Forms

  33. ELA Item Specifications 7

  34. ELA Item/Task Specifications • SBAC Content Specifications for ELA update • Inform the development of the Item/Task Specifications Tables and Forms • Samples: • Focus on grade 8 • Claims 1–4, different targets, item types, and depth of knowledge

  35. ELA Item Specifications Item Specification Tables • Format • Information provided

  36. ELA Item Specifications Tables

  37. ELA Item Specifications Tables

  38. ELA Item Specifications Tables

  39. ELA Item Specifications Tables

  40. ELA Item Specifications Item Specification Forms • The forms in which item/task writers will write new items/tasks.

  41. ELA Item Specification Forms

  42. ELA Item Specification Forms

  43. ELA Item Specification Forms

  44. ELA Item Specification Forms

  45. Mathematics Item Specifications 8

  46. Mathematics Item Specifications • How the Mathematics Content Specifications inform the development of the Item Specifications

  47. Mathematics Item Specifications • For Showcase 1, focused on one major domain in each grade band. • Grade 5 – Numbers & Operations: Fractions • Grade 8 – Geometry • High School – Algebra • Created specification tables for Claim 1 targets in those major domains. • Developed sample items for Claims 1–4 in the same domains.

  48. Mathematics Item Specifications • How the work is organized – • Read through the mathematics content specifications (a work in progress). • Created an Excel spreadsheet to capture, at a glance, the scope of work for each grade. • Teams of assessment specialists were assigned to work together (in grade bands) to develop Item Specification tables and sample items that illustrate our current understanding of the Claims, Targets, and best ways to assess them.

  49. Mathematics Item Specifications • Grade 4 at a glance • *Note: %’s represent allocation of sample items * * * *

  50. Mathematics Item Specifications

More Related