NH ALPs Science Scoring Training

1. NH ALPs Science Scoring Training July 2012

2. Training Outline • Science Learning Progressions • Levels of Scoring • Clarifications • Scoring – Forms, Template, Evidence • Video • Question & Answer Period

3. Science Learning Progressions Required Content Areas Gr. 4: Earth-Space Science & Life Science Gr. 8: Earth-Space Science & Physical Science Gr. 11: Life Science & Physical Science Each Entry Requires ONE Content Standard And TWO Process Skill Standards

4. How To Read The Progressions Content Standards • Challenge Level • Sub-Level • Column Process Skill Standards

5. Student as AuthorThe Critical Role of Accessto Learning, Communication & Performance What does “independent performance” mean for an alternately assessed student? Directive prompts violate the learning construct that we are trying to measure. No credit can be earned when directive prompts are used. Supportive prompts allow students to show personal authorship of standards-based tasks. Students can earn credit when supportive prompts are used because the constructs being tested are honored and demonstrated.

6. Two Levels of Score Review 1. Task Level Review • Content Fidelity 2. Item Level Review

7. Task Level Review Task Level Review determines whether or not the student performance sample can advance to the Item Level Score Review, based on sufficiency of match to the required academic content standards (Content Fidelity). If a student performance work sample is found not to sufficiently match the required content, the sample does not pass task review.

8. Task Level Review During this review, a scorer must ask the following two (2) questions: • Is the student attempting to apply knowledge or skills that are clearly within the content area? AND • Is the type of skills and knowledge clearly related to the standards of the progression?

9. Task Level Review continued… • If the answer to both of these questions is YES, then sufficient evidence of content knowledge exists and scoring may continue to the Item Level Review. • If the answer to either or both of these questions is NO, then sufficient evidence of content knowledge does not exist and scoring may not continue. If student performance does not meet this criterion, then the task sample cannot continue to the Item Level Reviewand the applicable comment code must be documented.

10. Task Level Review Comment Codes 1. Student method of communication could not be determined. Content Assessment Cover Sheet information was not submitted and Summary Description of Student Access to Learning and Performance was not submitted. 2. Challenge Level and/or Sub-Level was not identified. 3. Performance task evidence provided, as shown by student work sample, does not match content progression. (i.e.: Required Reading text type was not submitted, required Writing genre not submitted, required Mathematics progression not submitted, or required Science domain not submitted). 4. Copy or summary of text was not submitted for Response to Text writing sample. 5. The Science column could not be determined (last letter in the Science Content ALPs code).

11. Item Level Review An item is referred to as a scorable unit within a content standard. There may be one or multiple scorable units within a content standard. Each scorable unit is considered to be worth one (1) point.

12. ITEM LEVEL REVIEW 1. Is the performance criteria met as stated in the standard (or in the clarifications)? AND 2. Is the student clearly the author of this academic performance? • If the answer tobothof these questions is YES, then one (1) point is credited. • If the answer either or both of these questions is NO, then no point is credited. • If student did not earn credit for any scorable units within the chosen standard then it is considered Not Demonstrated (ND) and applicable comment code must be documented.

13. Item Level Comment Codes 6. Quality of video impacted score (i.e., sound quality, camera angle, clarity) 7. Video exceeded maximum allowable length. Score review ended at maximum time. 8. Student did not demonstrate standard(s), as written, in identified Progression or sub-progression. 9. Directive prompts prevented observation of student authorship. 10. Directive prompts impacted observation of student authorship. 11. Science Process Skill was not indicated. 12. Challenge level of Science Process Skill did not match Science content challenge level in this entry. 13. Science Process Skill was duplicated; only 1 Process Skill qualified for score review.

14. Question 1Is the Performance Criterion Met? Science success at the item level is defined as one clear demonstration of a science scorable unit - as written that also includes clear evidence of student authorship. “One clear demonstration” means that student performance must meet the scoring criterion defined by the scorable unit(s) with the standard- as written.

15. Question 2 Is the student clearly the author of this academic performance? Credit or non-credit of the scorable unit is also determined by the demonstration of student authorship. There must be compelling evidence that the student is clearly the author, as demonstrated in his/her performance of the scorable units (or item responses) within the standard.

16. Student as Author The following questions may serve as guidance to a scorer judgment in making the determination regarding the presence or absence of student authorship: • Does evidence of student access to learning, communication, and performance show student engaging in the task as a willful agentwho is effectively communicating his/her decision-making to indicate personal choice and authorship of performance of this scorable unit/item? • Did the teacher provide supportive prompts and not directive prompts? (See Training Manual) • Is there a biasing response context?

17. Student As Author continued… • Is the student able to communicate choices effectively? • If a student has a severe motor impairment and some responses appear to be a result of the clear willful intention of the student but some responses seem possibly accidental or random, is evidence of student response control sufficiently clear to make authorship compelling? (Requires scorer judgment)

18. Terminology Used in Science Progressions Understanding the specificity of the Science scorable units as they are written. The words “And” or “&”when placed between SUs of a standard requires the student to demonstrate one SU ORthe other SU, demonstration of both is NOT necessary to receive credit for one or the other. However, when the word “and” is used to connect elements INSIDE a single scorable unit of a standard, then both elements linked by the word “and” ARE required, (i.e. “student must match, produce, and compare”) requires demonstration of allelements in the group connected by the word ‘and’ are required before credit can be earned for this scorable unit. Demonstration of only one is not sufficient for credit when a scorable unit is written this way.

19. Terminology continued… • “Or” - when used to link a group of elements within a scorable unit, requires the student to demonstrate either one or the other element that is connected by the word ‘or’ (minimum of one is required). • “Describe” means using an effective communication method, student gives an account of someone or something, including relevant characteristics, qualities, or events. Demonstration of the Process Skills will show full breadth of student’s knowledge, allowing scorer to give credit for all scorable units. Sample Standard Earth-Space Science (E3D) “ Describe the Sun as a star. (1 SU) (S:ESS2:4:1.2)”

20. Terminology continued… •  “Explain” means todefine; expound; to offer reasons for or a cause of; justify and make an idea clear using your own words or through use of a valid communication system. Demonstration of the Process Skills will show full breadth of student’s knowledge, allowing scorer to give credit for all scorable units. Examples include describing causes, effects or relationships; or identifying steps in a process. Specific Examples found in Science standards include: • “Explain what happens when heat is applied to an ice cube;” or • “Explain how falling water can be used to produce electricity;” or • “Explain the stages (steps) an open field goes through over time to change into a forest.” Sample Standard Earth-Space Science (E2D) “Explain that people should not look directly at the Sun because it is dangerous and may cause injury to the eye (1 SU)(S:ESS2:2:4.3) ”

21. Terminology continued… • “Sort or Classify” at Emergent or Beginning Challenge Levels meansdefined and /or structured templates are provided by teacher. The term “Sort” is the preferred term for earlier challenge levels. • “Sort or Classify” atTransitional, Intermediate, and Advanced Challenge Levels requires the student to demonstrate more independent sorting and classifying behavior - with more student initiated categorizations beginning at Transitional Challenge Level. The term “classify” is usually the preferred term at higher challenge levels. (See also ‘Open Sort’ below.) • “Open Sort” requires the student to determine sorting category independently, working at a higher challenge level, without teacher template sample. Student independence in sorting/classifying increases as challenge levels increase. Open sort is consistent with higher challenge levels.

22. Terminology continued… • Words that meana minimum of at least two (2) performance instances are required:“all”, “most”, “many”, or other words that use the plural form(s). • “Pairs” is plural and means more than one pair, requiring minimum of two (2) pairs, (vs. ‘Pair’ = 1 pair.) • “Variety” means two (2) or more. • Commas (,) when used to link a group of elements, are treated as ‘OR’ statements – if there are only commas and no words connecting the elements in the group, (i.e. “student must match, produce, compare”). This statement, as written, uses only commas and therefore, requires performance of only one (1) of these elements.

23. Terminology continued… • “Recognizing” (a) requires naming or clearly choosing the correct example from an array of options, Or, (b) demonstrates understanding of specified concept through proper usage. • “Distinguish” means to differentiate between (See wording of standard for specific distinction intended.) Student must demonstrate that he/she knows the difference between two things, indicating; that which is an example of something, as compared to something else that is not. • “Precise” means the language used is highly specific in a way that is appropriate to the content being written about. Precise terms for measurement might mean to specify units of inches, feet or meters; etc.

24. Clarifications General Scoring Clarifications for Science • Content-Process Skills Connection Rule: Student must make the connectionbetween Science Content Progression and Process Skills Progression by using the process skills to explore and/or perform the content standard identified.Credit will not be given for demonstration of Process Skills performed in isolation, without the connection to content. • More Than 2 Process Skills Rule: Only the first two Process Skills demonstrated will be scored. • Student Authorship Rule for Sorting: Student will be credited with authorship of “Sorting” even if teacher provides a defined/structured sorting template and other relevant evidence of student authorship is present.

25. Clarifications continued… • Cross-Over Standards, defined: Cross-over standards are duplicate standards that span across more than one Learning Progression; (i.e. same standard is found in both Earth-Space Science, and Physical Science Progressions.) Although the standards replicate the text language, the content will guide scoring approach. Student performance must always be demonstrated in relation to specified content progression being sampled in order to receive credit. Sample Standard This exact standard, (S:ESS1:2:7.1) “ Recognize that water can be a liquid or a solid; and explain that it can be made to change from one state to the other, but the amount (mass) of water always remains the same in either state. (1 SU)” is found in two Science Content Progressions, as follows:

26. Clarifications continued… Earth-Space Science Content Progression: Beginning Challenge Level • (EarthMaterials Sub-Progression) (B3B)Student performance must demonstrate understanding of standard as it applies to the Earth-Space Science content progression (i.e.: earth science student work sample will have to clearly connect with earth materials- water found in air, found in dirt, streams, etc.) (S:ESS1:2:7.1) And in:   Physical Science Content Progression: Beginning Challenge Level • (PropertiesofMatter Sub-Progression)(B1B)Student performance must demonstrate understanding of standard as it applies to the Physical Science content progression (i.e.: physical science student work sample will have to clearly connect with properties of matter more broadly than just earth materials - water found in synthetics, manufactured, or other materials, etc.) (S:ESS1:2:7.1)

27. Clarifications continued… Cross-Over Standard Scoring Rule: Student work sample must connect in a distinctly identifiable wayto the stated content standard to be eligible for credit. There are two possibilities: • If teacher stimulus provides same exact task activity for both Earth-Space Science and Physical Science Entries, the duplicated task that does not distinctly connect to Earth-Space Science or to Physical Science will not be scored. • If teacher stimulus provides same general activity for both Earth-Space Science and Physical Science Entries, BUT the student doesdemonstrate a distinct and specific connectionto the identified Earth-Space Content AND also specifically to the identified Physical Science Content, then student work sample can receive credit for both Entries.

28. Clarifications continued… Clarifications for Specific Science Progressions Life Science Death and Dying: Due to a possiblebias concern related to death and dying, credit will be given even if student does not demonstrate a full understanding or provide an explanation of “die” or “death” words as related to this progression. Knowledge and understanding of life cycle as a whole will receive credit. For Example: Life Science • E1E Recognize that living things have a life cycle, during which they are born, grow, and die S:LS1:2:3.2 N (1 SU) • B2E (1) Recognize that living organisms have life cycles, which include birth, growth and development, reproduction, and death; and (2)explain how these life cycles vary for different organisms S:LS1:4:3.2 N (2 SUs)

29. Clarifications continued… Process Skills Emergent Challenge Level EPS8 “(1) Sort and classify objects, materials and events based on one or more attributes; and (2) explain the methods used for sorting (2 SUs)” (S:SPS1:2:1.5) • Credit will be given if student demonstrates sort and classify object materials OR • Events based on one or more attributes. They do not have to demonstrate objects, materials and events (as identified by the word “and” placed inside this first scorable unit). They must only demonstrate sorting and classifying of one of these elements. • The second scorable unit (2) in this standard stands on its own and must be met as written to be credited.

30. Scoring Sample

31. Scoring Materials • Student Score Form • Step by Step Scoring Procedures for Science • Scoring Comment Codes • Training Manual: • Learning Progressions • Glossary

32. Scoring Samples

33. Questions