Science Alliance

1. Science Alliance November 29, 2007 Please submit your MC Question with your next step suggestions.

2. Group Norms • Start and end on time • Put cell phones on silent • Be respectful of all comments • Everyone participates • Exercise the rule of “two feet” • Come prepared for the meeting • Keep side conversations to a minimum

3. Review from October • Completed deconstruction of standards addressed in your unit • Developed a performance task • Experienced a vocabulary/engagement strategy • Examined an assessment plan for congruency between stages 1 and 2 • Deepened understanding of FA

4. Agenda for Today • Continue Stage 2 Development • Rubric Development • Peer Review • FA Strategy • Quality MC and ORQ’s • DoK Levels • Item Development • Self Assessment of Alignment • Vocabulary Strategy

5. “In times of change, the learners will inherit the world, while the knowers remain well-prepared for a world that no longer exists.” • Eric Hoffer, writer

6. Performance Assessment • There are two parts to a performance assessment: • The Task • Simple target – simple task • Complex target – complex task • The Performance Criteria • We use performance assessments when the learning targets require doing (e.g., design, produce, make, create, make, write, draw, represent, display, model, construct). • Performance assessments are ways to allow students to demonstrate how well they can do science – not the drills, but the game!

7. Performance Assessment • The task is any activity that we use as a context to observe a skill or a product – a naturally occurring event or a separate event, at the end of instruction or during instruction. The only requirement is that the task elicits the desired skill or product so that it is capable of being judged. Pg. 194 CASL • …performance criteria [is] the basis for judging the quality of the performance on the task. Pg. 194 CASL

8. Benefits of Performance Criteria • To help educators clarify the nature of complex learning targets so that they feel comfortable teaching to them • To assess student progress and status in ways that are consistent across students, assignments, and time • To improve student achievement by letting students in on the secret of the nature of quality • Through all these things, to integrate assessment and instruction and grasp the essence of standards-based instruction • Scoring Rubrics in the Classroom by Arter and McTighe, pg. 16

9. For Teachers: Consistency in Scoring Improved Instruction For Students: Clear Target(s) Quality Performance Self Assess and Adjust Benefits of Performance Criteria

10. Rubrics • A rubric is a particular format for criteria – it is the written down version of the criteria, with all the score points described and defined. The best rubrics are worded in a way that covers the essence of what we, as teachers, look for when we’re judging quality, and they reflect the best thinking in the field as to what constitutes good performance. Rubrics are frequently accompanied by examples (anchors) of products or performances to illustrate the various score points on the scale. • Scoring Rubrics in the Classroom by Arter and McTighe pg. 8

11. Carousel Brainstorming – Performance Criteria • As a table group, brainstorm what performance criteria should do for us in the classroom. Capture your ideas on chart paper. 5 mins. • Pass your brainstormed list to another table, add to their list any other ideas not included. 3 mins. • Exchange charts one more time and add ideas not listed. 3 mins. • What can you generalize about performance criteria? What are the implications for your performance task?

13. Carousel Brainstorming – Rubric Features • As a table group, brainstorm the features a rubric should have to serve the performance criteria purposes. Capture your ideas on chart paper. 5 mins. • Pass your brainstormed list to another table, add to their list any other ideas not included. 3 mins. • Exchange charts one more time and add ideas not listed. 3 mins. • What can you generalize about features of rubrics? What are the implications for your rubric?

14. What Kind of Rubric? • Working with a partner, write your definition for each type of rubric listed and give an example of what type of assignment would be appropriate for each. • Jot down some disadvantages for each type. • Sketch an example for each. • Share with your tablemates and revise your ideas as needed.

15. What Kind of Rubric? • Identify the type of rubric that would best serve your needs and your student’s needs. • Holistic • Analytic • Generic (General) • Task-specific • Does it need to be unique to your task or is it one that could be used across similar performances (and thus best developed as a department or school team)?

16. Common Problems with Instructional Rubrics • Counting – when quality is more important than quantity • Important details left out • Irrelevant details in the rubric • Student-developed rubrics where anything goes • Skimpy scoring guides • Converting performance standards to grades

18. Developing Rubrics • Establish a knowledge base • Gather samples of student performance • Sort student work by level of quality • Cluster the reasons into traits • Identify sample performances that illustrate each level • Make it better • Pg 211 CASL

19. Work Time • Complete/revise your performance task using the criteria on pg. 220 in CASL and the GRASP format from UbD workbook pg. 172 as guides. • Develop the rubric for your task. Use the Metarubric Summary on pg. 203 in CASL and pgs. 181-195 in the UbD workbook as resources for development.

20. Peer Review of Performance Task and Rubric • Identify the PoS and/or CCA standard(s) being addressed by the performance task. • List what you consider the most obvious/important knowledge, reasoning, and skills that will be required by students to successfully complete the task. • Use the Task Rubric Summary on pg. 220 in CASL to provide feedback to the developer(s) concerning the task.

21. Peer Review of Performance Task and Rubric • Use the Metarubric Summary on pg. 203 in the CASL book as well as common problems with instructional rubrics discussed earlier (pgs. 205-210 in CASL) to provide feedback to the developer(s) concerning their draft rubric. • Meet with the developer to discuss strengths and weaknesses of the task and rubric.

22. Agenda for Today • Continue Stage 2 Development • Rubric Development • Peer Review • FA Strategy • Quality MC and ORQ’s • DoK Levels • Item Development • Self Assessment of Alignment • Vocabulary Strategy

23. Research-based Strategies • 4 Research-based strategies that significantly improve student learning: • Sharing criteria (clear learning targets with success criteria) • Questioning • Feedback • Peer and self-assessment

24. “More effort has to be spent in framing questions that are worth asking; that is, questions which explore issues that are critical to the development of students’ understanding.” • Assessment for Learning: Putting It Into Practice • Pg. 42

25. “You understand it only if you can teach it, use it, prove it, explain it, or read between the lines.” Wiggins and McTighe Understanding by Design “Current classroom practices generally encourage superficial and rote learning, concentrating on recall of isolated details, usually items of ‘knowledge’ which pupils soon forget.” Black and Wiliam Inside the Black Box Effective Questioning

26. Questioning should be used to: Cause thinking Provide data that informs teaching Questioning should NOT be used to: Direct the attention of the class Keep students on task Dylan Wiliam, NSTA Preconference on Assessment, 2005 Effective Questioning

29. Thinking Hats • Draw a hat from the baggie. • Find 2 others in the room with the same hat. • Consider the following question from the perspective specified by your hat color: Does human activity affect Earth’s ecosystems, both locally and globally? • Return to your original table. Have each person share their “hat” perspective pertaining to this question.

30. Six Thinking Hats • White Hat – facts, figures, information • Black Hat – caution, truth, judgment • Red Hat – emotions, feelings, hunches • Yellow Hat – advantages, benefits, good • Green Hat – create, explore, new ideas • Blue Hat – reflections, lacks decisions Formative Assessment in Action, p. 66

31. “If the only tool you have is a hammer, you tend to see every problem as a nail.” • Abraham Maslow

32. Agenda for Today • Continue Stage 2 Development • Rubric Development • Peer Review • FA Strategy • Quality MC and ORQ’s • DoK Levels • Item Development • Self Assessment of Alignment • Vocabulary Strategy

33. Key Resources for Development • Academic Expectations* • Core Content for Assessment* • Test Blueprint* • Kentucky Performance Level Descriptions • Content resources * These three play a key role in item development.

34. General Item-Writing Guidelines • Alignment to Standards • The match to Core Content for Assessment is essential to the validity of the KCCT. • Content Relevancy • Make sure the item assesses important knowledge or skills identified by standards in the Kentucky Core Content for Assessment.

35. Sample Item The Preamble and the Articles of Confederation were written in which year? A. 1697 B. 1765 C. 1787 D. 1865

36. Sample Item Which document describes the purpose of the U.S. Constitution? A. Preamble B. Bill of Rights C. Articles of Confederation D. Declaration of Independence

37. General Item-WritingGuidelines • Grade-Level Appropriateness • Make sure the item reflects the Core Content for Assessment at the appropriate grade level. • Basic vs. Technical Vocabulary • Use simple, basic vocabulary instead of technical vocabulary unless you are assessing the students’ knowledge of the meaning of the technical word/phrase.

38. Sample Item Science, Grade 4 When you plant a seed, the roots grow downward. This is called geotropism. Which factor is responsible for geotropism? vs. When you plant a seed, the roots grow downward, and the stem grows upward. Which factor is responsible for the roots growing downward?

39. General Item-Writing Guidelines • Grade Appropriate Vocabulary • Use grade-appropriate vocabulary as much as possible. Check the vocabulary lists to determine the grade level of words being used in the item. • Essential Information Only • Make sure that the item only includes information essential to understanding and answering the item. Text that is not essential to the item increases the reading burden on the student.

40. General Item-Writing Guidelines • Clear, Correct, and Understandable Graphics • Include clear, correct, easily understood graphics as required by the item. Do not use a graphic if it is not really essential to answering the item. • Bias and Sensitivity • Consider Bias and Sensitivity Guidelines when drafting items.

41. Guidelines for Developing Multiple-Choice Items

42. Advantages of Multiple-Choice Items • Can be used to measure a wide variety of learning outcomes • Permit wide sampling and broad coverage of a content domain • Are reliable and efficient to score • Can provide useful diagnostic information about the learning of individual students or groups of students

43. Disadvantages of Multiple-Choice Items • Multiple-choice items are difficult to write. • Multiple-choice items cannot measure certain types of skills (e.g., the ability to organize and express ideas in writing). • Performance on multiple-choice items can be influenced by student characteristics unrelated to the subject of measurement, such as reading ability and “test-wiseness.”

44. Guidelines for Developing Multiple-Choice Items Tips for writing response options… • Make the incorrect response options plausible. • Make sure one response option (especially the correct answer) does not stand out as being different from the other options. One option should not be much longer or shorter than the others, should not be different grammatically, etc.

45. Guidelines for Developing Multiple-Choice Items More tips… • In science and math items, consider using common misconceptions or errors as response options. • When appropriate, use other concepts or terms from the grade-appropriate Core Content for Assessment.

46. Guidelines for Developing Multiple-Choice Items More tips… • Avoid humorous or nonsensical response options. • Avoid using the options “all of the above” and “none of the above”.

47. Guidelines for Developing Multiple-Choice Items More tips… • Logically order the response options that include numbers, dates, etc. Numbers should be listed in ascending or descending order. Unless testing the sequence of events from a passage, list options in the order in which they appear in the passage.

48. Training Examples for • Multiple-Choice Items

49. Do all four response options make sense? Each response option should make sense both grammatically and with respect to content.

50. In order to grow and flourish, pumpkins need lots of A. milk, eggs, and molasses. B. sun, water, and space.* C. gourds, melons, and cucumbers. D. cream, eggs, and lots of sugar and spices.