Using Assessment to Improve Teaching and Learning

1. Using Assessment to Improve Teaching and Learning Florida Association of Science Supervisors Assessment Workshop with Karen Hollweg Jacksonville, FL October 15, 2003

2. A Conceptual Map for Investigating the Influence of Nationally Developed Standards for Mathematics, Science, and Technology Education from Weiss et al. (Eds), 2002, Investigating the Influence of Standards: A Framework for Research in Mathematics, Science, and Technology Education. Washington, DC: National Academy Press.

3. YOUR RESPONSIBILITIES What? Who?

4. Learning Cycle Engage Apply Explore Evaluate Propose explanations

5.  Where are we trying to go?  Where are we now?  How can we get there?

6. SCIENCE ASSESSMENT

7. NSES Teaching Standard C (p. 37-38) • Teachers of science engage in ongoing assessment of their teaching and of student learning. In doing this, teachers • Use multiple methods and systematically gather data about student understanding and ability. • Analyze assessment data to guide teaching. • Guide students in self-assessment. • Use student data, observations of teaching, and interactions with colleagues to reflect on and improve teaching practice. • Use student data, observations of teaching, and interactions with colleagues to report student achievement & opportunities to learn to students, teachers, parents, policy makers, & the general public.

8. SCIENCE ASSESSMENT

9. Black and Wiliam’s Review Overview Reviewed studies re: formative assessment (i.e., when the evidence is actually used to adapt teaching to meet student needs) published worldwide 1989-97 Used the following criteria to select studies: • Matched experimental and control groups • Pre and post tests • Frequent assessment used as feedback to guide learning • Quantitative analysis to compare learning gains

10. Black and Wiliam – continued Findings • Between 20 and 30 relevant studies found • All show significant learning advantage with formative assessment • Effect sizes range from 0.4 to 0.7. • Several studies show that “low attainers” show the largest gains • Several involve emphasis on self- and peer-assessment by pupils Most of these studies lacked detail of the classroom methods and criterion tests used. YET, they show a variety of ways of enhancing formative assessment, indicating that the gain effect is robust.

11. Standards are raised only by changes that are put into direct effect by teachers and students in classrooms. There is a body of firm evidence that formative assessment is an essential feature of classroom work and that development of it can raise standards. We know of no other way of raising standards for which such a strong prima facie case can be made on the basis of evidence of such large learning gains. (pg. 19) Black and Wiliam, 1998

12. Looking Inside the Classroom • Horizon Research (2003) • …fewer than 1 in 5 mathematics & science lessons: • Are strong in intellectual rigor; • Include teacher questioning that is likely to enhance student conceptual understanding; and • Provide sense-making appropriate for the needs of the students and the purposes of the lesson. See full report at www.horizon-research.com/insidetheclassroom

13. Public's Attitudes Toward The Public Schools • Phi Delta Kappa/Gallup Poll (2003) • 69% “lack information” on the NCLB Act • 84% believe the job a school is doing should bemeasured on the basis of improvement shown by students. • 72% believe it is NOT possible to accurately judge a student's proficiency in English and math on the basis of a single test. • The 35th Annual Phi Delta Kappa/Gallup Poll of the Public's Attitudes Toward The Public Schools”, Lowell C. Rose and Alec M. Gallup, Phi Delta Kappan, September, 2003, Volume 85. See http://www.pdkintl.org/kappan/k0309pol.htm

14. ElCentro Schools Is Implementing NSF-funded Instructional Matls with PD …Results After 4 Yrs. Amaral, O., Garrison, L. and Klentschy, M. (Summer 2002). Helping english learners increase achievement through inquiry-based science instruction. Bilingual research Journal, 26:2, 213-239.

15. El Centro (CA) School District 4th & 6th grade student scores also showed similar increases in reading and math SAT scores.

16.  Where are we trying to go?  Where are we now?  How can we get there?

17.  Where are we trying to go? Students should develop  an understanding of properties in matter… “A substance has characteristic properties, such as density … independent of the amount of the sample.” (p.154) abilities necessary to do scientific inquiry… “…develop explanations & predictions using evidence – both evidence that does & does not agree with current beliefs/concepts” (p. 144-145) “…thinking critically about evidence – accounting for anomalous data… & form a logical argument” using the evidence. (p. 145)

18. Where are the students now? • What have they done? • What do they appear to be able to do and understand? •  How can we get there? • What question would you pose next for these students? • What additional learning opportunities would you suggest?

19. Butler – On Feedback Overview • 132 students aged 11 from 12 classes in 4 schools • Selected top 25% and bottom 25% for data analysis Experiment • Divided students into 3 groups (within each class): A, B, C • All students given same teaching by two teachers • All students given same aims and criteria, asked to produce the same work for marking

20. What do you think happened? • On your worksheet • Indicate the % gain or loss in the 2nd column, • And • Note whether student attitudes became more positive or negative in the 3rd column

21. Butler – continued Findings Butler, Ruth, 1988. Enhancing and undermining intrinsic motivation: The effects of task-involving and ego-involving evaluation on interest and performance. British Journal of Educational Psychology, 58, pp. 1-14.

22. White & Frederiksen – On Students Overview • 12 science classes in two schools, ages 12-13 • New course on Forces and Motion (Thinker Tools) • Practical work with emphasis on open/inquiry projects • Criteria for marking explained

23. White & Frederiksen – continued Experiment Two matched sets of classes with the laboratory and associated work the same for all, except for one part: • The “experimental” classes had reflective discussion about assessing their work in relation to understanding science, understanding inquiry, team-work and communication. • The “control” classes used similar time to discuss the design of the teaching and to formulate suggestions for ways to improve it. For all classes: • CTBS skills at outset • Three teachers mark all projects, overall and each student’s selected best • Pre and Post tests of physics concepts, inquiry skills, attitudes about science and science learning

24. What do you think they found regarding the performance of the experimental groups in comparison to the control groups on…? • Projects • Post test results • And • Note whether you think some students changed more or less (specify which students)

25. White & Frederiksen – continued Findings • Experimental group (reflective assessment) performed better on projects and tests that assessed physics concepts, inquiry skills, and attitudes about science and science learning • Performance improved more significantly for the “lower” achieving students (as based on CTBS score) White, B.Y., & Frederiksen, J.R. (1998). Inquiry, modeling and meta-cognition: Making science accessible to all students. Cognition and Instruction 16(1), 3-118.

26. The Student’s Essential Role “The student is the one who must take action to ‘close’ the gap between what they know and what is expected (Sadler, 1989). A teacher can facilitate this process by providing opportunities for participation and multiple points of entry, but students actually have to take the necessary action.” from pg. 48 Classroom Assessment and the National Science Education Standards (NRC 2001)

27. What are the implications for practice based on this evidence?

28. Look at Inquiry 4.1, • Identify specific concepts & responses you will want to see in student work. • Look at Student Sheet 4.1 copies, • Identify student work that you could use as exemplar(s) • Write 1-2 feedback sentences (comments) related to the learning goals/standards that you would give to students.

29. One condition that promotes adult learning is "careful and continuous guided reflection and discussion about the proposed ideas or changes.“ (S. Mundry, et al., Designing Successful Professional Meetings and Conferences in Education, 2000)