1 / 51

FASS Conference 2006

FASS Conference 2006. FCAT Updates Linda Oravetz FCAT Science Coordinator Test Development Center Florida Department of Education. Topics . FCAT Science News Test Design & Cognitive Complexity Science & School Grading Calculations Test Item Specifications, Appendix B

ayala
Download Presentation

FASS Conference 2006

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. FASS Conference 2006 FCAT Updates Linda Oravetz FCAT Science Coordinator Test Development Center Florida Department of Education Florida Department of Education

  2. Topics FCAT Science News • Test Design & Cognitive Complexity • Science & School Grading Calculations • Test Item Specifications, Appendix B • Test Releases 2007, 2008 • Sample Test Materials • Florida Inquires & CD Training Materials • SSS Revisions Impact Plan • Other Interpretive Products Florida Department of Education

  3. More Topics • FCAT Development Summary • FCAT Scoring Process Summary • Other FCAT Information Florida Department of Education

  4. FCAT Science Achievement Levels Scale Scores Florida Department of Education

  5. Revised School Grading Rule The school grading scale will be adjusted in 2006-07 to account for the addition of the Science, and Mathematics Lowest 25 Percent components, adding 200 points to the scale. Florida Department of Education

  6. Science and School Grades One point will be awarded for each percent of students meeting high standards by scoring in Achievement Level 3 or higher on FCAT Science Florida Department of Education

  7. FCAT Science 4 Reporting Clusters • Physical & Chemical Sciences (Strands A–C) • Earth & Space Sciences (Strands D–E) • Life & Environmental Sciences (Strands F-G) • Scientific Thinking (Strand H) Florida Department of Education

  8. Test DesignGrades 5, 8, and 11 Counting Toward Student Scores: • Approximately 25% points from each reporting cluster • Annually assessed benchmarks • Content sampled benchmarks • A performance task from each reporting cluster; total 3 SR, 1 ER • Multiple-choice items • Gridded-response items (grades 8 & 11) Florida Department of Education

  9. Item Types and Numbers Range includes operational and field-tested items: • Grade 5 45-55 MC, 5-7 performance tasks • Grades 8 and 11 40-45 MC, 3-6 GR, 5-7 performance tasks Florida Department of Education

  10. Test Design Percentage of points by cognitive complexity: • Low 15 - 25 • Moderate 40 – 60 • High 25 – 35 Florida Department of Education

  11. Cognitive Complexity Low Complexity Test Items Require • Recall and recognition • Retrieving table or chart information • Solving a problem using reference sheet Florida Department of Education

  12. Cognitive Complexity Moderate Complexity Test Items Require • Describing or explaining • Comparing and/or contrasting • Determining the next step Florida Department of Education

  13. Cognitive Complexity High Complexity Test Items Require • Analyzing • Drawing conclusions • Generalizing • Proposing method for correcting flaws Florida Department of Education

  14. For More Information • www.firn.edu/doe/sas/fcat/fcatpub2.htm • www.fldoe.org • Test design summary • Content focus reports • Cognitive Complexity • FCAT Rubrics • And much more Florida Department of Education

  15. Item Specifications • 2002 Draft FCAT Item Specification still on DOE website • Current plans to add Addendum to that draft to include revised Appendix B, Rubrics, & Reference Sheets • Meanwhile – Appendix B available by email

  16. Florida Department of Education

  17. 2007 Fall Releases Science Grades 5 and 8 2007 Operational Tests Grade 11 will be released in 2008 Florida Department of Education

  18. Released Tests • The released tests are formatted to remove field-test items. All items on which scores were based are released. • The released tests are reformatted to fit into one document, so answer spaces and grids are with each item. http://www.firn.edu/doe/sas/fcatrelease.html Florida Department of Education

  19. Released Tests Information that is provided with test items: • % students who correctly answered the item • % students who chose each distractor (option) for MC items • Example of top score response for each performance task • Score point distribution for performance task items Florida Department of Education

  20. Released Tests Other information provided with each test item: • Benchmark • Content focus • Cognitive Complexity Florida Department of Education

  21. SSS Revisions Impact Planning

  22. SSS Revision’s Impact on FCAT • After the new SSS are approved by the State Board of Education, the FCAT Content Advisory Committee will meet to discuss changes needed to FCAT based on the new standards. • Technical Advisory Committee will review changes and make recommendations to the Department. Florida Department of Education

  23. Minor SSS Revisions • Recoding of items currently in item pool • Updating of Item Specifications • Application of new item coding to development • Application of new benchmark criteria to development • No change to test itself Florida Department of Education

  24. Significant SSS Revisions After approval of new SSS: New FCAT Item Specifications prepared for Item Writer Training. Development begins on next-generation FCAT items. Two years after approval of new SSS: First cycle of next-generation FCAT item development complete. Test construction process begins for following year’s test. Florida Department of Education

  25. FCAT Interpretive Products Science Related Products

  26. Deliveries Fall Sample Test Materials Spring Florida Inquires! Florida Department of Education

  27. 2006 Training Materials to Districts CD of training materials from 2006 released short response item • Mathematics, Reading, Science, Writing+ • Rangefinder (Anchor) set- 10 papers • One Qualifying set – 10 papers • Annotations (score and scoring notes for both sets) • Quantities – 2 per district, one per school Florida Department of Education

  28. Watch For • FCAT Explorer – Grade 5 Science Florida Department of Education

  29. Registered trademark ® FCAT Products and Resources contain the state seal to indicate that they are published by the Florida Department of Education. The Florida Comprehensive Assessment Test ® is now a registered trademark. FCAT Products are sent to the District Coordinators of Assessment for dissemination. Florida Department of Education

  30. FCAT Development and Scoring • Development Contractor – Harcourt Assessment Incorporated • Administration and Scoring Contractor – CTB/McGraw-Hill Florida Department of Education

  31. Development Related Committees • Bias Review – October • Community Sensitivity Review – October • Content Review – October through November • Expert Review - January Florida Department of Education

  32. Item Development Pilot Test Content Review Bias Review FieldTest Statistical Review Expert Panel Review Sensitivity Review FCAT Development Process (Science only) Florida Department of Education

  33. Scoring Committees • Gridded-response Adjudication – April • Rangefinding – April through May and September • Rangefinder Review – November through December Florida Department of Education

  34. FCAT Handscoring Process Florida Department of Education

  35. Handscoring Quality Controls • Florida educators set scoring criteria • Scorers hired by contractor are required to have college degrees in related areas in order to be eligible for training. • Scorers receive high-quality training under the supervision of DOE. • Scorers must qualify to score by demonstrating accuracy and consistency in matching scores set by educators. Florida Department of Education

  36. Handscoring Quality Controls • Scoring is monitored by DOE staff (on-site for first training and first weeks of scoring). • All responses are scored independently by two scorers. “Discrepant” scores are resolved with a third scoring by scoring supervisor or leader. • Scorers falling below threshold are retrained and/or dismissed. Florida Department of Education

  37. Other FCAT Committees • Assessment and Accountability Advisory Committee • Subject Area Content Advisory Committees • Interpretive Products Advisory Committee • Technical Advisory Committee • Computer-Based Assessment Advisory Committee Florida Department of Education

  38. FCAT Committees • Prompt Review Committee • Standards-Setting Committees • Special Ad Hoc Committees Committee participation by nomination Florida Department of Education

  39. FCAT Committees 2005-2006 • 677 participants • 38 weeks of meetings • 76 different meetings • Science meetings – 133 participants • Bias & Sensitivity – 78 participants • Advisory Groups – 9 Florida Department of Education

  40. A New Way for Reporting Scores The Parent Network

  41. FCAT Parent Network Florida Department of Education

  42. Student Scores Available on Parent Network for 2007 • Mathematics • Reading • Science • Writing+ Florida Department of Education

  43. FCAT Interpretive Products More things to look for

  44. FCAT Handbook— A Resource for Educators • Program-level information about the FCAT • Audience: EDUCATORS AND ADMINISTRATORS • Delivered to districts October 2005, and posted to DOE website Florida Department of Education

  45. Lessons Learned Contents • Narratives on what students are doing well in for each strand, what students are struggling with, and the implications for classroom instruction Florida Department of Education

  46. Lessons Learned Contents • Graphical presentation of student performance by Strand and by Achievement Levels, with analyses • Instructional implications based on input from a committee of Florida educators Florida Department of Education

  47. Lessons Learned for Science ?? • Anticipate work to begin on Lessons Learned for FCAT Science next fall • Estimated Fall 2008 release • Writing+ Lessons Learned to be on same development track as Science • Reading and Mathematics – 2007 Release, second volume Florida Department of Education

  48. Department of Education Web Address - www.fldoe.org • FCAT Publications (General Public) • FCAT Publications (Educator) • FCAT Scores and Reports • FCAT Developmental Scores Florida Department of Education

  49. Other FCAT Resources • FCAT Web Address -http://www.firn.edu/doe/sas/fcat/fcatpub2.htm • Test Design Summary • provides details about the structure of the test • Content Focus Reports • For each reporting cluster • More specific description of content measured Florida Department of Education

  50. For the Latest Information Join the Florida Department of Education Paperless Communication System at: www.firn.edu/doe/menu/communications.htm Florida Department of Education

More Related