Lessons Learned: Writing Collection of Evidence 2007 Scoring Results Implications for Instruction Steve Pearse, Ed.D.

1. Lessons Learned:Writing Collection of Evidence2007 Scoring Results�Implications for Instruction Steve Pearse, Ed.D.Writing COE SpecialistCAA Options / OSPI Washington State Assessment Conference

2. Writing COE Scoring Results 2 Guiding Principle for the COE�and for Scoring: ��only objective assessments that are comparable in rigor to the state assessment [WASL] are authorized as an alternative assessment.� �ESSB 6475, Sec. 1

3. Writing COE Scoring Results 3 Results of COE Writing Scoring � 31 Writing Collections met sufficiency requirements and were scored as per on-demand WASL Writing responses (expository + persuasive). � Each Collection was scored twice, with any non-adjacent scores triggering a third score; the two scores were then added to arrive at a total score. � Collections were scored holistically�across all Work Samples�for content, organization & style (COS) and for conventions (CONV). � As for Reading, we were unable to set standard this year, due to the small number of COE submissions. � The Writing Standard Setting Panel determined that seven (7) Collections were proficient*. * see document, Writing Working Descriptions of Performance�Grade 10

4. Writing COE Scoring Results 4 Which Collections Met Proficiency? � Proficient Collections align with Proficient Writing Descriptors, as listed in Writing Working Descriptions of Performance�Grade 10, March 2004. � Proficient Writing Collections� holistic scores�across all Work Samples�meet or exceed WASL standard as per aligned WA Writing Grade 10 Baseline Anchor Set responses. � Proficient Collections incorporate a variety of tasks/prompts, often including authentic, classroom-based assignments. � Each of the seven (7) proficient writing collections originates from a different Washington state school district.

5. Writing COE Scoring Results 5 Which Collections Did NOT Meet Proficiency?(continued) � These Writing Collections fall below 9, the cut line the Standard Setting Panel established for proficiency this year. � Non-Proficient Writing Collections reflect Basic Writing Descriptors, as per Writing Working Descriptions of Performance�Grade 10, March 2004. � Non-Proficient Writing Collections align holistically with below-standard Anchors used to score the WASL. � Non-Proficient (Basic) Collection Work Samples often reflect a lack of emphasis on process.

6. Writing COE Scoring Results 6 Which Collections Did NOT Meet Proficiency? � Persuasive responses among these Basic Collections often lack a sense of energy and/or purposefulness. � Formulaic organization and/or ineffective introductions and conclusions characterize many Basic Collection Work Samples. � These Basic Collections frequently reflect careless editing�or an apparent lack of attention to writing conventions (CONV). � Many Basic Collections include Work Samples whose topics or prompts seem inappropriate for or unhelpful for student-writers.

7. Writing COE Scoring Results 7 Lessons Learned � Purposeful instruction is essential to student success on the writing COE. �Many Collections� Work Samples and documentations of process suggest minimal instructional support. � Carefully-chosen topics or tasks (prompts) often correlate with effective writing. �Uninspiring, misleading, overly general, and/or narrowly-focused prompts align with an apparent lack of commitment to topic. � Engaging in a purposeful process contributes to writing performance. �Many Work Samples reflect �missed opportunities,� suggesting the potential to meet proficiency but failing to do so from initial work (pre-writing) to final drafts. �Little evidence of process characterizes many extended-time as well as on-demand Work Samples.

8. Writing COE Scoring Results 8 Lessons Learned (continued) � Taking advantage of word processing and attendant software (e.g., Spell Check) contributes to a higher level of writing performance. �Many Writing COE student-writers failed to do so, as evidenced by minimal or overly general Work Sample process documentation and by final drafts that were largely unchanged�and unimproved�from initial draft(s). � Research-based prompts and assignments should invite student-writer interpretation and analysis. Assignments or prompts that invite plagiarism place student-writers at risk of not presenting their own work. �Some Work Samples incorporating research reflect a lack of personal commitment (voice), focus and approach (ideas & organization), and/or expression (sentence variety & word choice).

9. Writing COE Scoring Results 9 Augmented Collection Requirements � The cut score for proficient Writing Collections was 9 points. � To be eligible for augmentation, Writing Collections must have earned a score of 8. � Eligible students MUST submit 3 additional Work Samples: one persuasive sample, one expository sample, and one additional�either persuasive or expository�sample.

10. Writing COE Scoring Results 10 

11. Writing COE Scoring Results 11 Writing Conventions (CONV) Scoring Criteria

12. Writing COE Scoring Results 12 Sample COE Writing CollectionsSample COE Writing Collections� Representative Basic (Non-Proficient) COE Collection: �G5� (Green binder, Collection #5)Score: 8 (below cut line of 9) (Scorers independently scored this Collection as a 3,2 for COS and as a 2,1 for CONV = 8 total scoring points.) This Collection consists of six (6) Writing Samples, all but one of which responding to Grade 10 WASL expository or persuasive prompts administered in previous years. Although the student-writer chose to provide drafts as evidence of process (as opposed to explanations),final texts reflect only minor revisions of initial work. The followingWork Samples typify the Collection as a whole�

13. Writing COE Scoring Results 13   Work Sample #3: �Memorable Class� On-Demand / Expository Prompt: By the time you are a sophomore in high school, you have been in many different classes. Think about a memorable class. Write a multiple-paragraph letter to a teacher explaining why that class was memorable. [Source: WASL expository prompt, 2002]

14. Writing COE Scoring Results 14 �Memorable Class� Dear Teacher, My name is [ ] and I�m writing to you to tell you my most memorable class during my sophmore year. I had many fun classes that were memorable because I have memories from one class. My most memorable class was Deca. Deca was my most memorable class during my sophmore year. It is my most memorable class for many reasons. One of thease reasons was because me and my friends had it together, so we had a lot of fun. Me and my friends had a lot of fun because we got to work in the ramshack together and listen to music. Another reeson Deca is my most memorable class is because you got to do many cool thinks in there. Like we had to do real job interviews with the teacher. And dress for success. We would get to do presentations in front of the class advertising something that we had come up with and try to sale it.

15. Writing COE Scoring Results 15 �Memorable Class� My best memory from this class was (flc) fall leader ship confrences. When you get to go out of town and stay in a hotel for two nights, you get to meet business people and get samples of what they sele. you get shirts for memories. Therefore Deca is my most memorable class for all these reasons. it was the best class. it was a good experience and memory. There were many good memorable things in that class during my sophmore year.

16. Writing COE Scoring Results 16 Work Sample #3: �Memorable Class� Analysis The following Basic Level Writing Descriptors apply to this work sample: � Concrete thinking, limited ability to distinguish between an idea and supportive detail� � Little expansion of ideas and elaboration of details�repetition of ideas� � �basic formulaic organization�basic, often ordinal, transitions between paragraphs � �Kidspeak� (unable to�or doesn�t think to�adjust language for audience and purpose � Limited revision; haphazard editing� � Fragments and run-on sentences

17. Writing COE Scoring Results 17 Work Sample #3: �Memorable Class� Analysis(continued)

18. Writing COE Scoring Results 18 Work Sample #6: �Starting Time�Extended-time / Persuasive  Prompt: Many people have expressed concern about the starting time for high school. The school board has suggested that school begin two hours later and end two hours later. Take a position on this proposal, and write a multiple-paragraph letter to the school board persuading them to agree with your position. [Source: WASL persuasive prompt, 2002]

19. Writing COE Scoring Results 19 �Starting Time� Dear school board, My name is [ ] and Im writing you to explain why school shouldnt start two hours late. There are many reasons why school shouldnt start two hours late and thoose reasons are all about responsibility and the future. you will be amazed with why school sholdnt start late. you will agree and not think it over twice. My first reason for school not to start two hours late is because students are lazzy. The schools already make everything so easy on students and giving students more freedom than they deserve wouldnt be right. It will probably increase the number of car accidents and schools dont need there teenagers to be giving them a bad name. Keeping the time how it is now will help the students at your school suceed. because in the future they will have real jobs where they cant wake up two hours late.

20. Writing COE Scoring Results 20 �Starting Time� My last reason why I think school shouldn�t start two hours late is because students need to be more responsible with how they use there time. If you were to start school two hours late then the students will party till later and be home till later. That wouldnt be right, so it will be better if we kept our normal schedule so that students will be tired and realise they need to be more responsible and go to bed earlier. Therefore these are my reasons for school not to start two hours late. So that there could be dicipline for there future. and they not that when they get jobs there boss isnt going to say lets start work two hours later. Sincerely

21. Writing COE Scoring Results 21 Work Sample #6: �Starting Time� Analysis The following Basic-Level Writing Descriptors apply to this Work Sample: � Concrete thinking, limited ability to distinguish between an idea and supporting detail; � Blurred understanding of the difference between writing to explain and writing to persuade��persuasitory� � Some evidence of a plan�repetition of ideas � Little expansion of ideas and elaboration of details� � �basic formulaic organization�basic, often ordinal, transitions between paragraphs � �Kidspeak� (unable to�or doesn�t think to�adjust language for audience & purpose) � Limited revision; haphazard editing

22. Writing COE Scoring Results 22 Work Sample #6: �Starting Time� Analysis (continued) The writer of this Work Sample engaged in the most basic of writing processes: � a simple topic-based web/cluster, limited to single clauses or phrases (e.g., �need to stop being laazy�); � an initial draft that, aside from removing a sentence, is nearly identical to the final text; no evidence of reordering, rewording, or providing additional details or elaboration; � Although Writing Sample #6 comes closer to meeting CONV proficiency than does Sample #3, its final text retains conventions errors that could have easily been eliminated via minor editing, including misspellings (�suceed,� �laazy�) usage and sentence formation errors.

23. Writing COE Scoring Results 23 Sample COE Writing Collections� Representative Proficient COE Collection: �O1� (Orange binder, Collection #1)Score: 10 (above cut line of 9) (Scorers independently scored this Collection as a 3,3 for COS and as a 2,2 for CONV = 10 total scoring points.)  This Collection consists of six (6) Writing Samples, including responses to local school-related issues, classroom assignments, and personal-interest topics. Rather than include actual process documents, the student-writer provides often-detailed explanations of process undertaken in support of purposeful revising and editing. The following Work Sample profiles typify the Collection as a whole�

24. Writing COE Scoring Results 24 Work Sample #1: �My Little Brother�s Birth�On-demand / Expository Prompt: Write a personal essay entitled �A Life Changing Experience.� You must show the reader what you have learned and how it has changed your perspective. Your only requirement is to [practice] �SHOW� writing, and to be as honest as possible (without revealing anything that might make you uncomfortable).

25. Writing COE Scoring Results 25 �My Little Brother�s Birth� Five years ago, my entire family had a huge life changing experience; my little brother was born. It doesn�t sound like a big change but what happened was a true miracle. This even was life changing because it affected everyone differently; each person was changed for the better. My mom is diabetic, so for her to even have children is really challenging. The reason it is so hard for my mother is due to the regulating of her sugar blood level. It becomes very difficult and at times uncontrollable while carrying a fetus. She was willing to monitor her daily diet so carefully that her required insulin was limited. This made it possible for the fetus to survive. I admire her courage to restrict her diet to save a life. The day my brother was born was a sunny spring Sunday. I woke up very happy because I realized I had missed church, not my favorite activity. Actually it was the last place I wanted to be. I went downstairs to enjoy some TV and my sister came down stairs and told me that our mom was having the baby. The rest of the family was already at the hospital. I know I should have been worried, but I am not a real emotional guy, and I felt more excitement than fear.

26. Writing COE Scoring Results 26 �My Little Brother�s Birth� My sister drove too fast, she was anxious. When we arrived at the hospital, my little brother was already born. Unfortunately, there was a problem, he was sick. He did not have enough sugar in his blood. This is a very dangerous condition for a newborn and we were advised that he had a slim chance of surviving. I felt sad but useless. Fortunately, with the miracle of medicine he survived. I can remember the tension and how nervous we were that first week. The days seemed like an endless desert. Why he has been such a miracle to all of us is that before his birth we just didn�t understand how precious life was. Like most teenagers we took things for granted. Additionally, how important this life was to our parents as well. We learned the true value of unconditional love. We learned how to support one another. We learned how become a family. One life, one simple life changed us into better people and most importantly unified us as a family.

27. Writing COE Scoring Results 27 Work Sample #1: �My Little Brother�s Birth� Analysis The following Proficient-Level Writing Descriptors apply to this Work Sample: � Connection to topic; communication of basic ideas although demonstrates some original thinking � Adequate control of topic; some courage or risk-taking � Understanding of the difference between main idea and supporting details � Adequate elaboration�often layered � Adequate sentence-to-sentence progression within paragraphs � A logical organization pattern � Introductions sometimes pull the reader into the main idea or position; writes conclusions that leave the reader with something to think about � Adequate�sometimes sophisticated�transitions which serve to connect ideas between paragraphs � A sense of the person behind the words (voice); evidence of the writer�s own life in the communication (emotional connection) � Adequate word choice�sentence variety � Consistent use of a dictionary and/or Spell Check, and appropriate use of thesaurus � Attention to editing; consistent application of the rules for capitalization, usage, punctuation, spelling; non-distracting use of conventions

28. Writing COE Scoring Results 28 Work Sample #1: �My Little Brother�s Birth� Analysis (continued)

29. Writing COE Scoring Results 29 Work Sample #4: �Stay, Fight or Cut & Run?�Extended time / Persuasive Prompt: There are many opposing views and opinions regarding the war in Iraq. Choose a specific side to persuade another student answering the following question: Do we �Stay and Fight or Cut and Run?� Write a multi-paragraph essay which takes a specific side on the war in Iraq. Your writing should be in the persuasive mode with clear defense and references of your view point.

30. Writing COE Scoring Results 30 �Stay and Fight, or Cut and Run?�[excerpted] The conflict in Iraq has been one of the key reasons the United States our government and its constituents are in debate and turmoil. One cannot turn on the news without hearing people complain about the war in Iraq. Their complaints focus on the increasing casualities, deaths, money being spent to finance the war. Some people are literally sick over the war. Our congress has said if we cannot accomplish our original goal and finish our task of a government that can survive on its own. If that cannot be done in Iraq we will �cut and run.� This means that we will cut our loses and resources and leave Iraq for good. Since over half the population is against the war not many are fighting it. That being said I believe that The United States of America has never retreated for any war, why would we start now? Should we leave and act like it is not our problems. Or stay and fight for democracy. Clearly, we should stay and fight for the implementation of a true democracy and the sustaining of human rights for the Iraq people. By leaving before this task is put in place we are jeopardizing hundreds of innocent�s lives and quality of life. There are two sides that are in the middle of this conflict in Iraq. There are pro-war, and the anti-war. National Security adviser S. [states] �Congressional Democrats� efforts to put time limits on the Iraq war could doom U .S. military efforts and leave the strife-torn country a hub of international terrorism.� S indicates: �That if we leave Iraq regardless of what you believe in this could affect world-wide terrorism�

31. Writing COE Scoring Results 31 �Stay and Fight, or Cut and Run?�[excerpted, continued] The other side of the arguments is the anti-war side. There are people that my have not been against the war at the beginning, but because of the length, money, and the number of deaths people are wanting for the troops to come home. Seventy-three percent of America want our troops to come home. This is a hard position for the President�to convince the people of America to want to stay involved because of the high number of lives lost. He has to choose between the countries dissatisfaction of his stance in the war, and finishing what he thinks is right, the mission in Iraq. This pushes the pro-war people to try and fight to stay over there. The cause of leaving may put a profound effect on the people of Iraq and worldwide terror. Do the ends really justify the means?� The one main reason we went over into Iraq is that we knew that they had [ties] with al quida, and the fact that they were known and believed to have weapons of mass destruction. Even the ehad of the CIA M.�said that his Intel said that there were weapons of mass destruction. We went over there to stop

32. Writing COE Scoring Results 32 �Stay and Fight, or Cut and Run?�[excerpted, continued] the spread of terror. We did not find any WMD there is a lot of speculation of what happened to them but the Intel said otherwise. This is the reason that we should stay over there and finish what we have started. As a woman said in an interview with CNN, �We need to fight for people that could not fight for themselves. My position is clear�we absolutely need to stay and fight for the rights of Iraqis�. It is our duty as Americans to uphold civil rights. We are a country that believes in the democratic process and we cannot leave until this is in place and fully functioning. If we leave now we would be leaving this country in the same circumstances and situation we initially found it in. This would be a great injustice and a waste of time, money and lives. Keeping our commitment guarantees the protection of Iraqi�s future generations. We might not be able to stop terrorism; but we can give the Iraqis� a foundation for peace and equal rights for all its citizens.

33. Writing COE Scoring Results 33 Work Sample #4: �Stay and Fight, or Cut and Run?� Analysis

34. Writing COE Scoring Results 34 Work Sample #4: �Stay and Fight, or Cut and Run?� Analysis (continued) This writer�s explanation of the writing process used to produce this response takes advantage of COE requirements and opportunities, as evidenced by a clearly proficient (especially for COS) final draft. The student�s explanation emphasizes the power of authentic classroom learning and work in the service of COE Writing extended time response performance: �This was an extended paper done in several phases with some teacher assistance. We have been working on the Persuasive Essay this year and this was our culminating essay. We debated sides in class, took notes, went on the internet for research, watched the debates on CNN and then had to choose a side. This paper went through three drafts all typed because my handwriting is so poor. I did my best to show both sides but I am very clear that we should stay the course in Iraq. This was not a popular stance in my class. I felt this is one of my better essays.�

35. Writing COE Scoring Results 35 Writing Assessment Instructional Support Moduleswww.k12.wa.us/assessment/WASL/writing/writing � High School Persuasive Writing Module (Grades 8-10) Version 2 (June 2007) � High School Four-Week Course � Middle and High School Introductions and Conclusions � Middle and High School Elaboration [ plus ELL Appendices and Resources]

36. Writing COE Scoring Results 36 Certificate of Academic Achievement Options 2007-2008 Key Dates:The Collection of Evidence (COE) � 12/15: COE registration for February submission closes � 2/15: COE submission date to OSPI � 4/15: COE registration for June closes � 6/16: COE submission date to OSPI