Formative Assessment: A Method to Close the Feedback Loop

1. Formative Assessment: A Method to Close the Feedback Loop Eugenia Etkina, Graduate School of Education Rutgers University 2005 Winter AAPT Albuquerque, NM

2. Members of Rutgers ASA Project Alan Van Heuvelen, Sahana Murthy, David Brookes, Aaron Warren, David Rosengrant, Maria Ruibal Vilassenor, Suzanne Brahmia, Julia Timofeeva NSF ASA Program http://paer.rutgers.edu/scientificabilities/ http://paer.rutgers.edu/PT3

3. Outline • Assessment • Formative vs summative • Three steps of formative assessment • Self-assessment • Examples

4. Why do we need assessment? • One of the purposes of assessment within education is that of informing and improving students’ ongoing learning • Summative and formative

5. Formative Assessment • Formative assessment: the process used by teachers and students to recognize and respond to student learning in order to enhance that learning during learning. • Gains reported due to formative assessment are the largest reported for an educational intervention (Black and Wiliam).

6. Essential components of formative assessment • Teacher giving feedback to the students • The teacher and students taking actionto improve learning during learning • Self - assessment

7. Where do you need to go? Where are you now? How can you get there? Three essential steps

8. Self-assessment • Students must be be able to understand and use the criteria with which they are assessed, in order to bridge the gap between what they know and can do and the desired goal

9. What do we want to assess? • Conceptual understanding • Problem solving • Scientific abilities

10. What are some scientific abilities? • Ability to represent a process in multiple ways • Ability to design an experimental investigation (an observational experiment; a testing experiment; an investigation to solve a problem) • Ability to collect and analyze experimental data • Ability to construct and modify explanations • Ability to evaluate all of the above

11. Formative assessment tasksand rubrics Assessing students’ scientific abilities • Multiple representation tasks (D. Rosengrant, A. Van Heuvelen, E. Etkina) • Experimental design tasks (S. Murthy, E. Etkina) • Anomalous data tasks (D. Brookes, E. Etkina) • Video problems tasks (D. Brookes, E. Etkina) • Evaluation tasks (A. Warren, A. Van Heuvelen) http://paer.rutgers.edu/scientificabilities http://paer.rutgers.edu/pt3

12. Using rubrics Design two independent experiments to determine the specific heat of the given unknown object. Student writing samples … the block and water will reach equilibrium after 10 minutes … no heat goes in and out of the calorimeter… minimal heat is lost to the environment during transfer of the block … the temperature inside the beaker is homogenous SCORE: 3 … No heat exchange between system and surroundings, no temperature gradient inside. SCORE: 1

13. Using rubrics Design two independent experiments to determine the specific heat of the given unknown object. … if the heat is lost from the block during transfer, the specific heat obtained in the experiment would be smaller than what it should be. SCORE: 3 … the assumptions will affect the results, the actual value may be different from the experimental one. SCORE: 1

14. Development of rubrics • Identifying important sub-abilities • Writing descriptors (scale 0-3) • Finding clear wording • Scoring student work • Discussing the items with a disagreement • Revising wording

15. Where do we use them? • Lectures - electronic student response system wit peer interactions and instructor feedback • Recitations - interactions with peers and TA • Labs - interactions with peers, self assessment with rubrics,interactions with a TA • Homework - interactions with a TA, posted solutions • Exams (summative - research purposes)

16. a y a y F cable on elevator F cable on elevator F Earth on elevator F Earth on elevator Ability to represent phenomena in multiple ways Free-body diagrams: where do you need to go? An elevator is slowing down An elevator is slowing down on on its way up:its way down: Earth, cable

17. Rubric for self assessement Free-body diagrams: How to get there?

18. 1 2 4 Fperson 3 Fball on person Formative assessment task in lecture Where are you now? Which free-body diagram best represents the ball thrown In the air?

19. Do students actually use FBDs?

20. Problem solving strategy Where do you need to go • Picture and Translate • Sketch the problem situation; include all known information. • Choose a system object and make a list of objects that interact with the system. • Indicate the direction of acceleration, if you know it. • Simplify • Determine if you can ignore any interactions of the environment with the system object. • Represent Physically • Draw a free-body diagram for the system. • Represent mathematically • Apply Newton’s second law in component form to the situation you represent in the free-body diagram. • Add kinematics equations if necessary. • Solve and evaluate

21. Ability to evaluate somebody’s problem solving Where are you now? • The problem A 1000-kg elevator is moving down at 6.0 m/s. It slows to a stop in 3.0 m as it approaches the ground floor. Determine the force that the cable supporting the elevator exerts on the elevator as it stops. Assume that g = 10 N/kg. • Proposed solution The acceleration of the elevator is: • a = vo2/2d = (6.0 m/s)2/2(3.0 m) = 6.0 m/s2. • The force of the cable on the elevator while stopping is: • T = ma =(1000 kg)(6.0 m/s2) = 6000 N. How to get there? • Identify all missing elements in this solution • Identify any errors in this solution. • Provide a corrected solution if there are errors.

22. Ability to devise relationships and test them • Observe the fall of two objects dropped simultaneously. Describe your observations in words, with a motion diagram and mathematically. • What if? • Predict what will happen to the distance between two objects one of which was dropped slightly before the other: a) The distance will stay the same; b) the distance will decrease; c) the distance will increase; d) not enough information to answer. • Explain your prediction explain using words, motion diagrams, and mathematics. List assumptions. • Observe the experiment and revise your explanation if necessary.

23. Example of student reasoning The motion diagrams below represent the positions of the balls at every frame. The distance between them should increase. 1. * 1. * 2. * 2. * 3. `* 3. * 4. * 4. * 5. *

24. Rubric for self-assessment

25. Rubric for self-assessment

26. Ability to deal with anomalous data Real physics with formative assessment

27. Ability to deal with anomalous data Predict what will happen to the left bob (Nothing; will go up; will go down; will go up and down will swing left to right) • Explain your prediction using free-body diagrams. List additional assumptions.

28. Making a prediction Fstring on bob Fstring on bob Left bob Right bob FEarth on bob FEarth on bob

29. Revising the model or the assumptions • Observe the experiment and revise your explanation if necessary. • Decide whether you need to revise the model that you used to make a prediction or the additional assumptions.

30. The END Thank you!

31. Design observation experiment Design an experiment to determine if there is a relationship between the pressure and temperature of the gas inside a sealed hollow sphere. Equipment: Sealed hollow metal sphere with a pressure gauge, hot plate, thermometer. a) Describe your experimental design. Include the following: • How will you vary the temperature? How will you measure it? • How will you vary the pressure? How will you measure it? b) What other equipment do you need? c) Draw a labeled diagram of your experimental set-up. d) Record your observations in a table. e) What pattern did you find from your observations? What factors influenced the physical quantities you measured? f) Discuss whether your methods of measurement were reliable.

32. Design testing experiment Design an experiment to test the following rule: an object always moves in the direction of the net force exerted on it. You have a dynamics cart, dynamics track, a spring scale, masking tape, a bowling ball, a mallet, a small ball and a cushion to play with. You can also use any other common equipment available in the lab. Feel free to use your lab partner as an “object”! • State what rule you are going to test. • Brainstorm the task and make a list of possible experiments whose outcome you can predict. Decide what experiments are best. Briefly explain why. • Draw a labeled sketch of the experimental set-up and write a brief description of your procedure. • Draw a free body diagram of the object. • Make a prediction about the outcome of the experiment. Make sure that the prediction is based on the rule that you are testing. • Perform the experiment. Record the outcome. Was your prediction confirmed? • Based on your prediction and the outcome of your experiment, can you say that the rule is supported or not? .

33. Hypothetico-deductive reasoning

34. “We assume that the students who are aware that they have changed their beliefs and can justify this change on grounds such as greater explanatory adequacy, should be more capable of defending their beliefs from criticism and thus their learning should be less fragile”. [S. Vosniadou] .

35. Design an Investigation to Solve a Problem Design two independent experiments to determine the width of a strand of your hair. One method must involve ideas of diffraction. Equipment: Laser pointer, meter stick, holder for hair, screen, Vernier calipers.  Devise and write an outline of the procedure.  Draw a labeled diagram of your experiment.  Write the mathematical procedure you will use.  Write how you will measure the physical quantities you need.  List the assumptions are you making in your design. What are possible experimental uncertainties? How could you minimize them?  Perform the experiment. Record your measurements.  Calculate the thickness based on your procedure and measurements. Evaluate the effect of the uncertainties.  When finished both experiments, compare the two values for the thickness. What are possible reasons for the difference?

36. Anomalous Data Tasks • Use existing physics knowledge (tested and accepted) to make a prediction about an outcome of a particular experiment • Explain why you made the prediction. • Watch the experiment, record the outcome, compare to the prediction. • Revise the explanation or the assumptions if necessary.

37. Video Problems Observe the two experiments below. Use each to determine the “…”. • Describe how you will use the video to determine the necessary quantities. • List all physics explanations/relationships you will use to determine “…” • List all of the assumptions that you made. Describe the mathematical procedure that you will use to find the “…” using the measured physical quantities. • Decide whether you have a reasonable agreement between the results of the two experiments. Evaluate the assumptions and uncertainties.

38. Use of rubrics • To help instructors write assignments • To guide students writing lab reports • To help students with self-assessment • To help instructors provide feedback to the students • To help instructors grade student work

39. Future talks • S. Murthy - designing labs and assessing student experimental abilities • D. Rosengrant - designing multiple representation tasks and assessing whether students use multiple representations while solving multiple choice problems on the exams • A. Warren - designing evaluation tasks and assessing whether there is a relationship between student evaluation ability and problem solving ability.

40. Assess student work

41. Rubric for self assessment

42. Rubric for self assessment