A Look at Scoring MSA Science

1. A Look at Scoring MSA Science Ann Hermann aherrmann@msde.state.md.us Paul Katula pkatula@msde.state.md.us MAG Conference 2007 November 14, 15, 16 2007

2. Meeting Objectives • To discuss the Scoring process for MSA • To review the MSA science rubric • To develop an understanding of MSA science items • To review sample MSA science items with student responses • To discuss impact on instruction • Questions and answers

3. MSA Science Scoring Process • Students take the MSA science test. • Field test items go through rangefinding. • Scoring materials are developed using rangefinding responses. • Scoring directors prepare a Scoring Guide, 2 Training Sets, 3 Qualifying Sets, and Validity Sets. • Scoring occurs at the vendor’s site in Jacksonville, FL.

4. Each response is read by two scorers. • If they disagree by more than 1 score point a third read is done (0-2; 1-3) • Scoring directors and team leaders read behind all scorers. • MSDE scoring staff are onsite throughout scoring project. • Validity papers are sent through to readers as a check to be sure that all scorers are on target.

5. MSA Science Rubric • Defines each score point • minimal; general; full and complete • Provides a framework for what is expected • evidence; synthesis; terminology • Allows for a range of responses within each score point

6. MSA Science Rubric

7. Score Level 0 • There is evidence that the student has no understanding of the question or problem. • The response is completely incorrect or irrelevant or there is no response.

8. Score Level 1 There is evidence in this response that the student has minimal understanding of the question or problem. The supporting scientific evidence is minimal. The response provides little or no synthesis of information, such as data, cause-effect relationships, or other collected evidence. The accurate use of scientific terminology may not be present in the response. An application, if attempted, is minimal.* * On the Maryland School Assessment, the application of a concept to a practical problem or real-world situation will be scored when it is required in the response and requested in the item stem.

9. Score Level 2 • There is evidence in this response that the student has a general understanding of the question or problem. • The supporting scientific evidence is generally complete with some integration of scientific concepts, principles, and/or skills. • The response reflects some synthesis of information, such as data, cause-effect relationships, or other collected evidence. • The accurate use of scientific terminology is present in the response.

10. An application of the concept to a practical problem or real-world situation reveals a general understanding of the scientific principles. * • On the Maryland School Assessment, the application of a concept to a practical problem or real-world situation will be scored when it is required in the response and requested in the item stem.

11. Score Level 3 • There is evidence in this response that the student has a full and complete understanding of the question or problem. • The supporting scientific evidence is complete and demonstrates a full integration of scientific concepts, principles, and/or skills. • The response reflects a complete synthesis of information, such as data, cause-effect relationships, or other collected evidence. • The accurate use of scientific terminology strengthens the response.

12. An effective application of the concept to a practical problem or real-world situation reveals a complete understanding of the scientific principles.* *On the Maryland School Assessment, the application of a concept to a practical problem or real-world situation will be scored when it is required in the response and requested in the item stem.

13. Any questions about the MSA science rubric?

14. MSA Science Items • Grade 5 items are based on the VSC for grades 4 and 5 • Grade 8 items are based on the VSC for grades 6, 7, or 8 • BCR items will always have some type of stimulus or technical passage • Students must address the over-arching question. • Bullets help students address the over-arching question • Answer spaces are sometimes divided to help students organize their response.

15. Sample test items with student responses and scores

18. What is this question asking? • 4.D.1.b – describe what happens to the properties of materials when several materials are combined to make a mixture • Student needs to understand the properties of materials and what happens to these properties when they are mixed • Properties of materials • Shape, color, size, texture, taste, odor • Change in properties • Water – changed color, taste • Drink mix – changed texture, dissolved • Sugar – changed texture, dissolved

20. Score = 0 • Before the properties are mix they have different types of sodium and different chyrod (if it had any chyrod in it). It just had different backgrounds. • After the properties are mix they kind of have the same types of sodium, but at the same time, this mixer makes a different drink (not that anyone would drink it).

21. Score = 0 • Before the properties were mixed they were just water, sugar and drink mix. • After you mix everything you get a new beverage.

22. Score = 1 • Were dry and powdery. After they were mixed with water, they dissolved partially • Are dissolved partially and are no longer powder. They are wet crystals and give the water flavor.

23. Score = 2 • Water: clear and thin Drink mix: powder, colored and solid sugar: white and solid • Water: difrent color, taste and slightly thicker mix: has dissolved into water making water difrent taste sugar: switter

24. Score = 3 • The water is of course liquid before mixing but the powdered drink mix and sugar are both solids the water is clear the sugar is white and the drink mix is most likely colored. • When all ingredients are mixed the 2 solids are disolved into the water making it all liquid but the taste is still there. The drink mix gives it a little flavor and the sugar makes it sweet. The water is there to make it a liquid it all turns out to be a colored liquid

27. What is the question asking? • 3.D.1.b Explain that in all environments organisms with similar needs may compete with one another for resources, including food, space, water, air, and shelter. • How competition affects this ecosystem • The animals that directly compete for food sources • Why both plants and animals compete for water

28. Score = 0 • rodent eat sagebrush and the rodent get eat by rapter is just how get eat by how and what the animal eats

29. Score = 1 • The reason why plants and animals compete for water is because they need it to survive, and there isn’t enough rainfall. The sagebrush, and Prickly Pear cactus rely on the rodent. The rodent, lizard, and snake compete for the raptor and Badger. Also the lizard competes with the rodent for the snake.

30. Score = 1 The animals that compete for the same food source are that the snake and the badger compete for rodents and lizards. The raptor eats everything. The plants and animals compete for food because that is how they grow, get their nutrients and reproduce.

31. Score = 2 On Earth, all living things, wether plants or animals, need water to survive. This is especially important in the desert where water is scarce. The raptor can obtain the rsources it wants easier than any other animal in its environment because it is not the prey of any animal. The other animals must also compete with each other to obtain food. Badgers, snakes, and raptors, for example, all eat rodents, but because the raptor is at the top of its food chain, it is most likely to ge that rodent.

32. Score = 3 Animals in this ecosystem compete for resources. First of all, the plants and animals all have to compete for water, without water, organisms cannot survive. Second, animals of different species must compete foor food. The snake, badger, and the raptor all must compete to eat the rodent and the rat. The raptor and the badger must compete to eat the snake. There is also competition between animals of the same species. For example, all of the raptors must compete against each other for the rodents, snakes, and lizards. In every ecosystem, each animal must compete to survive.

33. Technical Passages • Used to stimulate students prior knowledge • NOT used as a seek and find activity • Information from technical passage may be used to support response • Both SR and BCR items may be based on the passage

36. What is the question asking? • 1.B.1.b Offer reasons for findings • Requires an explanation of WHY the average times are different for the two objects. • To accomplish this, the bullets ask for information about the investigation, errors in investigation, how errors affect outcome, and how investigation can be improved.

37. Score = 0 • The errors in this investigation is wrong. These errors affected the out come because for the round shape they should have added the mass and the height it dropped. The investigation should be improved by adding the mass and the height the shaped dropped and, by dividing how many numbers there is and the numbers you added up to get the answer.

38. Score = 0 • The object shape was round and rectangle and the mass was 100 and 115 and The Height Dropped was 10,10. average of time to fall 10, 12.5

39. Score = 1 • The average times to fall were different because one object is round and the other is a rectangular and a object that is round can roll the fastest and a rectangular will just slid.

40. Score = 1 • The time was different because the shapes hases a different wieght from a round shape to a rectangular shape they both have different wieght

41. Score = 2 • The errors are one object is round and the other is rectangular and one has more mass so it will have a longer time to fall. It should be changed they should change the shape of one or both to make them equal in shape or equal in size & mass.

42. Score = 3 • The Average time to fall was different because the object had different shape. The error was that the objects were not the same shape. Rectangular objects fall at slower speeds because the flat bottom takes on more air. Next time, make both of the objects the same shape.

43. Score = 3 • The errors in this investigation are that there is 2 independent variable. One is the shape and the other is the mass. These errors affected the outcome because the mass doesn’t affect the time to fall, as said by the Free Fall article. The shape must affect it because they are different. One independent variable should be taken out to obtain valid data.

45. What is this question asking? • 5.A.1.c Compare accelerated and constant motions using time, distance, and velocity. • Requires a description of the two types of motion represented in the graph • Name the types of motion represented • constant; accelerated • Describe the difference between the two types of motion.

46. Score = 0 • Object A Graph - The type of motion is a straight line it shows the distance in meters and the time it took in seconds. • Object B Graph – The type of motion is also a straight line. It shows the velocity in meters per second and the time which is in seconds. The difference between the two types of motions are that Motion A is measuring distance and Motion B is measuring velocity.

47. Score = 1 • Object A Graph – the motion in graph A is moving faster as the time goes on which means that it covers more distance. • Object B Graph – The motion of the object in graph B shows me that the velocity of the object increases as time moves on.

48. Score = 2 • Object A Graph – The distance increases at a constant rate for object A so object A moves at a constant speed or velocity. • Object B Graph – The velocity of object B increases steadily as time goes by. So object B get faster at the same rate. This is different from object A because object moves at the same speed while object B keeps on speeding up at the same rate.

49. Score = 3 • The graph of Object A shows that the object is in constant motion, because as time increases, distance increases just as much. Object A is maintaining the same speed, not increasing nor decreasing in velocity. • The graph of Object B shows that the object is in accelerated motion, for as time increases, velocity increases. Object B is not maintaining the same speed, for the velocity is increasing. This means that the distance is becoming biger and bigger than the time. If the dependant variable for graph B was changed to distance the graph would appear to look like half of a U.