Assessment Update

1. Assessment Update Prepared for the Florida Organization of Instructional Leaders (FOIL) November 6, 2008 Cornelia S. Orr & Renn Edenfield Office of Assessment Florida Department of Education

2. Topics • Concordance Scores Update • Writing – Inappropriate Student Preparation • National Assessment of Educational Progress (NAEP) • Next Generation FCAT/Florida Standards Assessment and End of Course Exams • Impact of Budget Cuts • FTCE/FELE Program Updates

3. Concordance Scores Update • FCAT/SAT/ACT Concordant Scores for High School Graduation: No Changes The accessible data table that appears on this slide is available in the text only version. The Outline version does not support table formats. • FCAT/HSCT Concordant scores for high school graduation • Studies and Rule Development – In progress • HSCT to be discontinued in 2009

4. Concordance Update (cont.) • FCAT/CPT Concordant scores for college readiness • 2006 study – concordance scores were not recommended • Only moderate correlations—0.676 Reading, 0.611 Math • CPT/SAT/ACT Concordant scores for college readiness • Studied conducted in 2007-08 • No changes recommended at this point The accessible data table that appears on this slide is available in the text only version. The Outline version does not support table formats. http://www.fldoe.org/articulation/perfCPT/default.asp http://www.fldoe.org/articulation/perfCPT/default.asp

5. Florida Writes!

6. FCAT Writing Goal • To obtain a valid measure of each student’s writing proficiency in a 45-minute demand writing situation • To judge each student’s response against a common standard, providing a source of statewide information used to characterize writing performance of 4th, 8th, and 10th grade students

7. Writing Process The writing process is the progression of stages a writer goes through to create a product, from conception to birth. –Ruth Culham Process: Planning, Drafting, Revising, Editing, Publishing

8. Appropriate Instructional Practices Research cannot identify one single approach to writing instruction that will be effective with every learner because of the diverse backgrounds and learning styles of students…Still, current research on writing makes these things clear: Instructional practices, writing genres, and assessments should be holistic, authentic, and varied. James R. Squire Office of Policy Research ~ includes research-based recommendations for effective writing instruction and assessment: www.ncte.org. 2008 Policy Research Brief - Writing Now

9. FCAT Writing andInappropriate Instructional Practices Students are trained to write using the following template: • prescribed format for responding to any prompt, regardless of the topic, or intended purpose (mode) • practiced/memorized settings, words, phrases, elevated figurative language • contrived introduction - disconnected from the prompt or purpose for writing

10. Example: Grade 4 Template Phrasing • One ordinary day turned into an extraordinary day… • I ran as fast as wild fire. Then neon colors encircle me…POOF! • A kaleidoscope of colors encircled me…POOF! or BAM! • A potpourri of iridescent colors surrounded me…POOF!

11. More Template Wording – Grade 4 • In the blink of an eye, you will find me - a world of color swirled around my head. POOF! I am in dragon land, pirate land, fairy land, etc. • In the blink of an eye [Sally and I] were perambulating home from…POOF! • One quintessential, supersonic day… • Mango orange, mahogany brown, Paris pink, kiwi green, ruby red

12. Other Replicated Phrasing • A million dollars • As nervous as a marshmallow in a bonfire • Gossamer wings • C’est La Vie • Hasta la vista • Now you can clearly see…

13. Collectively – A Pattern Emerges! • Word for word phrasing – not a coincidence ~ rote memorization! • This practice distorts the authentic writing task and relegates the task to a type of “fill in the blank” exercise.

14. Is this cheating? Although the instructional method in question may not be defined as cheating in the traditional sense, it violates the explicit requirement that each student’s writing is his or her original work.

15. What’s wrong with this method? • If trained in the “template” method, students are prone to becoming “Stepford Writers.” • They may look like writers, act like writers, think they’re writers, but they are really task completers – practicing for FCAT Writing.

16. Is this the same as formulaic writing? • No, the 5-paragraph or other formulaic structures provide a basic scaffold for beginning or novice writers. • Writing is original to each student • The student should be able to remove the scaffold when ready. • Not an expectation for every writing purpose, but may be useful

17. Mode Matters • 2008 experiences identified a need for more information about and focus on the writing mode. • Writing instruction should include instruction on mode. • Two modes are spiraled in tests at each grade. • Scoring includes the evaluation of modes.

18. Purpose: The unmistakable purpose of this type of writing is to recount a personal or fictional experience based on a real or imagined event and to create a central theme or impression in the reader’s mind. Goal: The goal of this mode is to use insight, creativity, drama, suspense, humor, or fantasy to develop an identifiable story line (one that is easy to follow and paraphrase) through use of events and details that work together. Mode Matters Grade 4 – Narrative Writing

19. Purpose The unmistakable purpose of this type of writing is to inform, clarify, explain, define, or instruct. Goal The goal of this mode is a carefully crafted presentation of facts, examples, or definitions that create a clear, central focus and enhance the reader’s understanding; although objective (not dependent on emotion), the writing is often lively, engaging, and reflective of writer’s commitment to the topic. Mode Matters Grade 4, 8, 10 – Expository Writing

20. Purpose The unmistakable purpose of this type of writing is to convince the reader that a point of view is valid and/or that the reader should take a specific action. Goal The goal of this mode is a clearly stated topic, issue, or opinion with elaboration that indicates the writer’s understanding and conviction; the writer may refute counterarguments to clarify the position. Mode MattersGrades 8 and 10 Persuasive Writing

21. National Assessment of Educational Progress (NAEP) • January 26-March 3, 2009 • National/State sample of schools and students • Reading/Math/Science – Grades 4, 8, 12 • Science – Interactive Computer Tasks and Hands-on-Tasks only at certain schools • A few “national sample” schools will also have U.S. History, Civics, and Geography • High School Transcript Study – Electronic transcripts pilot with minimal onsite follow-up to access student cumulative records • District Assessment – Miami-Dade County Public Schools • http://www.fldoe.org/asp/naep/naep2009.asp

22. FCAT Transition Plans (11/08) The accessible data table that appears on this slide is available in the text only version. The Outline version does not support table formats.

23. FCAT High School Assessments Transition • FCAT Comprehensive Assessments • Reading Grade 9 and 10 • Performance tasks moving to grade 9 • Mathematics Grade 9/10 – changing content • Grade 9 – Algebra EOC Exam • Grade 10 – Geometry (75%) Number Operations, Probability, etc. (25%) • End of Course Exams • Algebra II Pilot in 2009 (6 districts: Brevard, Duval, Leon, Okaloosa, Sarasota, & Seminole) • State development of Algebra I and Biology test questions • Review of Published Exams, e.g., American History, Civics, Chemistry for more expedient implementation

24. 2008 Writing Legislation s. 1008.22, F.S. “Beginning with the 2012-2013 school year, the comprehensive assessments of writing shall be composed of a combination of selected-response test items, short-response performance tasks, and extended-response performance tasks, which shall measure a student's content knowledge of writing, including, but not limited to, paragraph and sentence structure, sentence construction, grammar and usage, punctuation, capitalization, spelling, parts of speech, verb tense, irregular verbs, subject-verb agreement, and noun-pronoun agreement.”

25. FCAT – Impact of Budget Reductions • No Writing MC questions/No NRT tests • No computer-based retake this year - paper only • No summer retake beginning with summer 2009 (will return in the future only as a computer-based test) • No Parent Network after Fall 2008 reporting is completed • No Certificates of Achievement • Removal of seals from several spring books • No screen reader (use read aloud accommodations) • No new Florida Reads! Writes! Solves! Inquires! (CD or paper)

26. New FTCE/FELE Services • Customer service center is open 7 to 7 Monday through Saturday. Most calls are answered immediately but wait times should not exceed three minutes. • All FTCE/FELE examinations are administered on CBT. Florida is the only state with this distinction. • Beginning January 1, 2009 score verification will be offered in ten CBT locations across the state.

27. FTCE Middle Grades Integrated Curriculum (MGIC) Examination • Issue: • MGIC certification (grades 5-9) allows teachers to teach Algebra Honors, Geometry Honors and Physical Science Honors high school credit courses.  • Teachers can earn this certification by passing the MGIC exam, but can miss all items of one core subject area—yet still be considered certified to teach all subject areas. • Actions: • Short term: Districts may require subject area certification (e.g., MG Math) • Long term: MGIC Competencies and Skills are currently being revised and this issue will be addressed.

28. Elementary K-6 FTCE Teacher Certification Examination • Issue: • The Next Generation SSS present a higher cognitive complexity level for students in grades K-6. • The Elementary K-6 exam (grades K-6) needs to be revised to reflect this higher cognitive complexity and to require the teacher to know more of the content domain for a passing score.  • Teachers must be equipped to give students a solid foundation for the middle school curriculum. • Actions: • Short term: Raise the cognitive complexity of the FTCE K-6 exam. • Long term: Conduct standard setting and restructure the exam scoring.

29. Elementary K-6 and MGIC Teacher Certification Exams – Two-Phase Plan The accessible data table that appears on this slide is available in the text only version. The Outline version does not support table formats.

30. FTCE/FELE Fee Increases and Registration Limits • Fee Increases: • General Revenue funding is used to underwrite the development and administration activities. • General Revenue funding results in lower test fees. • State revenue challenges prompted the Department to reevaluate fees. • SBE approved a fee increase for 2009 testing. • General Revenue funding will still underwrite a portion of the program activities. • Registration Limits: • Seating availability is limited at both computer-based and paper-based test sites. • The number of seats available/tests given is subject to the amount of funding allocated for the project. • There can be seasonal or periodic variations in demand for testing.

31. FTCE/FELE Fees • Effective January 1, 2009. The accessible data tables that appear on this slide are available in the text only version. The Outline version does not support table formats.