The LCSD APPR Story

1. The LCSD APPR Story The Local Assessment Chapter December 13, 2012

2. The Third Piece 15-20pts The straw that broke. . . The local assessment decision more NYS assessment an approved 3rd party assessment region or district developed Our own assessments we decide what learning is important diversify measures of student learning Measures student growth for SLOs too

3. Two Parts • I - Best practices in assessment development • II - Superintendent Approval Process • Assurances • Assessment Maps/Blueprints/Specs • Scoring Guides • Test Specifications • Testing Guides

4. Creating Summative Assessments Cheryl Covell Tompkins-Seneca-Tioga BOCES Model Schools Program

5. Accountability Based Assessment Instructionally Supportive Assessment • Summative • Formative • Private • Public • Infrequent • Frequent • Assess particular skills and content • Assesses broad standards • Validity most important • Reliability most important • Evaluate students at all levels • Evaluate if students are “meeting standard” • Sorts, sanctions, rewards • Informs curriculum and instruction

6. What kind of test? • Summative • Standardized • Coverage of the “most important” standards • Selected response and constructed response

7. Summative Assessment • Summarizes learning at a particular point in time. • Can be used to compare how well students mastered a set of learning goals.

8. Standards-based Assessment • Student performance is compared to a desired level of performance on the content. • Questions on the assessment are selected to cover important content. • Questions on the assessment are selected to match national, state or local content standards.

9. APPR Evidence – Considerations • How aligned and authentic are the assessment items to the learning content? • How valid and reliable are the assessments? • Are they verified as comparable and rigorous? • What, if any, administration accommodations must legally be made for students? • How are the assessments scored in terms of point values assigned per item and method of summarizing scores? • Have procedures been established to ensure those with vested interest do not score students’ assessments?

10. Selected Response Any item which asks students to select from several options. • Multiple choice • True/false • Matching • Fill in the blank, with word bank

11. Multiple Choice – Stems • Make the question as clear and direct as possible. • Make sure students know what you are asking by just reading the question – a proficient student should be able to come up with the right answer without first reading all of the possible choices. • Use negatives sparingly and emphasize them if used. • Which of the following choices is NOT a mammal? • Ask more than one question when students must first take in a lot of information.

12. Multiple Choice – Alternatives • Make sure there is only one unambiguously right answer. • Create plausible distracters. • Common misconceptions • Common mistakes • Technical jargon • Do not use emphasis (bold, capital letters). • Avoid repeating words from the stem. • Use “none of the above” with caution. • List choices in a logical order.

13. All of the following are correct procedures for putting out a fire in a pan on the stove except: a. Do not move the pan. *b. Pour water into the pan. c. Slide a fitted lid onto the pan. d. Turn off the burner controls.

14. All of the following are correct procedures for putting out a fire in a pan on the stove except: a. Do not move the pan. *b. Pour water into the pan. c. Slide a fitted lid onto the pan. d. Turn off the burner controls.

15. California: a. Contains the tallest mountain in the United States b. Has an eagle on its state flag. c. Is the second largest state in terms of area. *d. Was the location of the Gold Rush of 1849.

16. What is the main reason so many people moved to California in 1849? a. California land was fertile, plentiful, and inexpensive. *b. Gold was discovered in central California c. The east was preparing for a civil war. d. They wanted to establish religious settlements.

17. When conducting library research in education, which of the following is the best source to use for identifying pertinent journal articles? a. A Guide to Sources of Educational Information. *b. Current Index to Journals in Education. c. Resources in Education d. The International Encyclopedia of Education.

18. When conducting library research in education, which of the following is the best source to use for identifying pertinent journal articles? a. A Guide to Sources of Educational Information. *b. Education Index. c. Resources in Education. d. The International Encyclopedia of Education.

19. Suppose you are a mathematics professor who wants to determine whether or not your teaching of the unit on probability has had a significant effect on your students. You decide to analyze their scores from a test they took before the instruction and their scores from another exam taken after the instruction. Which of the following t-tests is appropriate to use in this situation? *a. Dependent samples. b. Heterogeneous samples. c. Homogeneous samples. d. Independent samples.

20. When analyzing your students’ pretest and posttest scores to determine if your teaching has had a significant effect, an appropriate statistic to use is the t-test for: *a. Dependent samples. b. Heterogeneous samples. c. Homogeneous samples. d. Independent samples.

21. How long does an annual plant generally live? a. It dies after the first year. b. It lives for many years. c. It lives for more than one year. d. It needs to be replanted each year.

22. How long does an annual plant generally live? a. Only one year. b. Only two years. c. Several years.

23. Constructed Response A constructed response requires students to use creativity, organization skills, and logic to develop an answer. • Essay • Short answer • Diagram • Performance • Fill in the blank – no word bank

24. Four Elements • Task • What students should do. • Evaluative Criteria • The essential ingredients of the skill being measured. • Quality Definitions • The description of how a student’s performance on each evaluative criteria would look at each scoring level. • Scoring Strategy • How a final score will be derived for the task.

26. Smarter Balanced Assessments

27. SmarterBalancedConsortium http://www.smarterbalanced.org/wordpress/wp-content/uploads/2012/05/TaskItemSpecifications/EnglishLanguageArtsLiteracy/ELARubrics.pdf Rubistar http://rubistar.4teachers.org/

28. Lansing Central School District Criteria and Assurances for Superintendent Approval Of Local End-of-Course Assessments Teacher(s) ________________________________ Course Name __________________________________ Assurances of Validity – Please check all boxes. 1. I have reviewed this assessment to determine that each item measures the learning target that I intend to measure. 2. I have reviewed this assessment to determine that, when taken together, these items assess curriculum that I have prioritized as important to students’ adult lives, important to students’ future learning, important to students’ learning in other content areas, or relevant to NYS Assessments. Assurances of Reliability – Please check all boxes. 3. I have reviewed this assessment, and I expect that this collection of assessment items will reliably assess student learning for the affiliated course. I.e., student scores on this assessment should correctly classify student learning (e.g., as developing, proficient, mastered, advanced). 4. I will immediately report any indications that student scores on this assessment do not reliably describe levels of student learning, if such irregular results occur. 5. This assessment includes scoring materials (e.g., answer key, rubric(s), points distribution for short answers, etc.) that support consistent scoring of all students’ assessments. Assurances of Bias Review – Please check all boxes. 6. I have reviewed this assessment for assessment bias and have taken all measures available to me to minimize bias in every item. Assurances of Rigor and Comparability – Please check box 7 and either 8 or 9. 7. I have reviewed this assessment and assure that it is aligned with the appropriate NYS Learning Standards, including grade-level aligned CCLS Literacy Standards for Social Studies or or for Technical Subjects. 8. I have reviewed this assessment for rigor, using the levels in Bloom’s Taxonomy, and assure that these items address a range of thinking skills, but most items require students to work above Bloom’s Comprehension level. OR 9. I assure that the assessment requires students to deeply comprehend complex grade level text (as defined in the CCLS) and to demonstrate that comprehension through informational or argumentative writing using text-based references. (Grades K-2 tasks may require narrative writing.) The signature(s) below indicate that the end-of-course assessment for the course listed above has been reviewed and revised for validity, reliability, bias, and rigor. Teacher Signature(s) ________________________________________ Date _____________________ ________________________________________ Date _____________________ The signature below indicates Superintendent approval for the end-of-course assessment listed above. Superintendent (or designee) Signature ______________________________________ Date _____________________

29. Validity and Reliability Validity = Accurate Inferences Reliability = Consistent Results

30. A Valid Assessment….

31. Assurances of Validity • I have reviewed this assessment to determine that each item measures the learning target that I intend to measure. • I have reviewed this assessment to determine that, when taken together, these items assess curriculum that I have prioritized as important to students’ adult lives, important to students’ future learning, important to students’ learning in other content areas, or relevant to NYS Assessments.

32. Standards • Statements of what we want students to know and be able to do.

33. Learning Targets • The specific knowledge and skills a student must have in order to master a given standard.

34. A Reliable Assessment….

35. Assurances of Reliability •  I have reviewed this assessment, and I expect that this collection of assessment items will reliably assess student learning for the affiliated course. I.e., student scores on this assessment should correctly classify student learning (e.g., as developing, proficient, mastered, advanced). • I will immediately report any indications that student scores on this assessment do not reliably describe levels of student learning, if such irregular results occur. • This assessment includes scoring materials (e.g., answer key, rubric(s), points distribution for short answers, etc.) that support consistent scoring of all students’ assessments.

36. For objective assessments like multiple choice tests, to increase reliability, you should: • Include enough items. • Allow enough time for students to complete the test. • Standardize administration.

37. For papers, essays, and projects, to increase reliability, you should: • Have clear enough directions for students that all are likely to produce work you can score. • Have a systematic scoring procedure. • Have multiple markers (scorers) when possible. • Use broad categories for classification. • Standardize administration.

38. Assurances of Bias Review • I have reviewed this assessment for assessment bias and have taken all measures available to me to minimize bias in every item.

39. Bias Anything in the assessment or way the assessment is carried out that might not allow students to be able to adequately demonstrate what they know and can do. • offensive language or topics • assumed background knowledge • requiring skills outside of the area being assessed • noise – typos, poor layout, confusing directions

40. Topics to Avoid • abortion • abuse of people or animals • contraception • deportation of immigrants • experimentation on people or animals that is dangerous or painful • killing of animals for sport • the occult, witches, ghosts, vampires • pregnancy of human beings • rape • sexual behavior or sexual innuendo • suicide • torture • euthanasia • gun control • climate change caused by human behavior • prayer in school • current or recent partisan political issues, ethnic conflicts, and religious disputes

41. Topics to be Treated with Care • Accidents and natural disasters • Advocacy • Alcohol, tobacco, and illegal drugs • Animals that are frightening to children • Antisocial, criminal, or inappropriate behaviors • Biographical materials • Dancing • Dangerous activities • Death and dying • Evolution • Family problems • Gambling • Holidays and birthdays • Homelessness and evictions • Immigration • Junk food • Luxuries • Medicines, including diet supplements • Obesity and body-image problems • Personal questions • Religion • Serious illnesses • Slavery • Terrorism, wars, violence, suffering

42. Looking for bias…. • Shaquan helps assemble food packages for poor people at Christmas. Each box holds 6 cans in a row. There is room for 4 rows in a box. Write the expression that best describes the number of cans in one full box. • Unacceptable. The first sentence adds to the reading load but adds no useful information. The references to “Christmas” and “poor people” are inappropriate and unnecessary.

43. Looking for bias…. • Lee’s father and Juan’s father are both policemen. • Unacceptable. Even though both officers are male, “police officers” is preferred to “policemen” to avoid the impression that only men are police officers.

44. Looking for bias…. • If one card is taken at random from a deck of playing cards, what is the probability that the card will be an ace? • Unacceptable. The question assumes knowledge of the number of aces and the total number of cards in a deck of playing cards. It is acceptable to ask about probability, and it is acceptable to use playing cards in math problems. According to the guideline about gambling, however, it is not acceptable to assume that test takers have knowledge of the characteristics of a deck of playing cards.

45. Looking for bias…. • When Ms. Luna pulled her car into the parking garage, the machine at the gate issued a ticket stamped with the time, 11:30 a.m. When she left the garage that afternoon, her ticket was stamped with the time she left, 12:15 p.m. What was the total length of time that Ms. Luna’s car was in the parking garage? • Unacceptable. The question is very wordy and uses an unfamiliar context for many children. In addition, “pulled her car” is an idiom that children may not know.

46. Looking for bias…. • People who drive gas-guzzling SUVs contribute to global warming. • Unacceptable. This excerpt is a clear violation of the guideline against advocating for one side in a controversial situation.

47. Assurances of Rigor and Comparability • I have reviewed this assessment and assure that it is aligned with the appropriate NYS Learning Standards, including grade-level aligned CCLS Literacy Standards for Social Studies or for Technical Subjects. • I have reviewed this assessment for rigor, using the levels in Bloom’s Taxonomy, and assure that these items address a range of thinking skills, but most items require students to work above Bloom’s Comprehension level. ****OR**** • I assure that the assessment requires students to deeply comprehend complex grade level text (as defined in the CCLS) and to demonstrate that comprehension through informational or argumentative writing using text-based references. (Grades K-2 tasks may require narrative writing.)

48. Rigor…

49. Rigor Definition – APPR Field Guidance Rigorous means that the locally-selected measure is aligned to the New York State learning standards or, in instances where there are no such learning standards that apply to a subject/grade level, evidence of alignment to research-based learning standards.

50. Common Core Learning Standard 7.RP - Ratios & Proportional Relationships “Use proportional relationships to solve multistep ratio and percent problems.”