Informative assessment -- understanding guiding learning

1. Informative assessment --understanding & guiding learning Margaret Forster This presentation explores informative assessment Informative assessment addresses the fundamental purpose of assessment � to understand where a student is in their learning in order to decide the best way to facilitate further learning; that is to use information gained through assessment to assist in the process of understanding and guiding learning. Informative assessment does not make a distinction between the contexts of assessment or their stated primary purposes. For example, it does not make a distinction between assessments that are sometimes labelled assessments of learning such as NAPLAN, and assessments sometimes labelled assessments for learning such as a teacher devised diagnostic test of fractions understandings This presentation explores informative assessment Informative assessment addresses the fundamental purpose of assessment � to understand where a student is in their learning in order to decide the best way to facilitate further learning; that is to use information gained through assessment to assist in the process of understanding and guiding learning. Informative assessment does not make a distinction between the contexts of assessment or their stated primary purposes. For example, it does not make a distinction between assessments that are sometimes labelled assessments of learning such as NAPLAN, and assessments sometimes labelled assessments for learning such as a teacher devised diagnostic test of fractions understandings

2. Informative assessment focuses on how teachers and students make use of assessment information from multiple sources to inform and to drive teaching and learning. � Rather informative assessment focuses on how assessment information from multiple sources is used by both teachers and students to inform and drive teaching and learning. -- note the deliberate emphasis on multiple sources and use by teachers and students. From the teacher�s perspective informative assessment can be characterised as purposeful pedagogy; from the student�s perspective as purposeful learning. Purposeful because the teaching and learning is driven by assessment evidence; what each student knows, understands and can do including partial understandings and misconceptions. Not by expectations based on age, grade or gender� � � Rather informative assessment focuses on how assessment information from multiple sources is used by both teachers and students to inform and drive teaching and learning. -- note the deliberate emphasis on multiple sources and use by teachers and students. From the teacher�s perspective informative assessment can be characterised as purposeful pedagogy; from the student�s perspective as purposeful learning. Purposeful because the teaching and learning is driven by assessment evidence; what each student knows, understands and can do including partial understandings and misconceptions. Not by expectations based on age, grade or gender� �

3. Informative assessment draws on the research that underpins �assessment for learning� including understanding of what it means to progress in an area of learning, on research on high performing school systems; and on research on highly effective teaching. And as Geoff suggested in his address this morning the research into learning can be summarised as four closely related findings. It is these four findings that provide the structure of this presentation � What implications do these findings have for the practice of informative assessment? What does assessment practice as a central driver for teaching and learning look like?Informative assessment draws on the research that underpins �assessment for learning� including understanding of what it means to progress in an area of learning, on research on high performing school systems; and on research on highly effective teaching. And as Geoff suggested in his address this morning the research into learning can be summarised as four closely related findings. It is these four findings that provide the structure of this presentation � What implications do these findings have for the practice of informative assessment? What does assessment practice as a central driver for teaching and learning look like?

4. Informative assessment draws on the research that underpins �assessment for learning� including understanding of what it means to progress in an area of learning, on research on high performing school systems; and on research on highly effective teaching. And as Geoff suggested in his address this morning the research into learning can be summarised as four closely related findings. It is these four findings that provide the structure of this presentation � What implications do these findings have for the practice of informative assessment? What does assessment practice as a central driver for teaching and learning look like?Informative assessment draws on the research that underpins �assessment for learning� including understanding of what it means to progress in an area of learning, on research on high performing school systems; and on research on highly effective teaching. And as Geoff suggested in his address this morning the research into learning can be summarised as four closely related findings. It is these four findings that provide the structure of this presentation � What implications do these findings have for the practice of informative assessment? What does assessment practice as a central driver for teaching and learning look like?

5. The first implication is that� This means that when we think about gathering evidence of students� learning we cannot begin with the assumption that a one-size fits all assessment will necessarily provide us with sufficient information to understand what each child knows understands and can do.The first implication is that� This means that when we think about gathering evidence of students� learning we cannot begin with the assumption that a one-size fits all assessment will necessarily provide us with sufficient information to understand what each child knows understands and can do.

6. When we think about equity issues and disadvantage we readily think about addressing the needs of students from different language, social and cultural backgrounds but less commonly think about the needs of students who are working at different levels of learningWhen we think about equity issues and disadvantage we readily think about addressing the needs of students from different language, social and cultural backgrounds but less commonly think about the needs of students who are working at different levels of learning

7. When we offer a student learning experiences that are too difficult, we put that student at risk of failure. When we offer a student learning experiences that are too easy, we put that student at risk of failing to progress at a rate appropriate for them. When we offer a student assessments that are far too difficult, or far too easy, we lose the opportunity to discover the skills, knowledge and understandings that they already have. That�s not to say that we receive no information. For example, if a student is unsuccessful on all the questions we ask in a test, we have some understanding of what they don�t know. If they get everything right, we have some understanding of what they do know but in both cases insufficient understanding about where they are in their own learning so that we can decided based on evidence what to do next. The point I am making here is that assessment should allow all students of all backgrounds and levels of achievement to demonstrate what the know understand and can do so that we are in a position to develop learning experiences appropriate to the needs of individual students When we offer a student learning experiences that are too difficult, we put that student at risk of failure. When we offer a student learning experiences that are too easy, we put that student at risk of failing to progress at a rate appropriate for them. When we offer a student assessments that are far too difficult, or far too easy, we lose the opportunity to discover the skills, knowledge and understandings that they already have. That�s not to say that we receive no information. For example, if a student is unsuccessful on all the questions we ask in a test, we have some understanding of what they don�t know. If they get everything right, we have some understanding of what they do know but in both cases insufficient understanding about where they are in their own learning so that we can decided based on evidence what to do next. The point I am making here is that assessment should allow all students of all backgrounds and levels of achievement to demonstrate what the know understand and can do so that we are in a position to develop learning experiences appropriate to the needs of individual students

8. As Ausubel said in 1968�As Ausubel said in 1968�

9. The second and related implication of learning as a personal process is that � I began by saying that informative assessment does not emphasise sources of information. It does, however, emphasise quality of information ; in particular the diagnostic power of an assessment. That is, the power to illuminate strengths and weaknesses in students� understanding at either the group level or the level of the individual student�so that interventions can be designed to improve learning. Different kinds of tests provide different levels of diagnostic power. For example, a numeracy test that addresses students� number, measurement, space, chance and data understandings (primarily for system monitoring purposes) cannot provide the same fine-grained information about students� understandings of fractions, as a tool designed specifically for that purpose. In their practice, accomplished teachers use diagnostic information from tools of different diagnostic power as they �drill down� to obtain detailed information about a student�s knowledge, skills and understandings. What does drill down assessment look like? It exposes how students are thinking about a problem as well as what they know. The second and related implication of learning as a personal process is that � I began by saying that informative assessment does not emphasise sources of information. It does, however, emphasise quality of information ; in particular the diagnostic power of an assessment. That is, the power to illuminate strengths and weaknesses in students� understanding at either the group level or the level of the individual student�so that interventions can be designed to improve learning. Different kinds of tests provide different levels of diagnostic power. For example, a numeracy test that addresses students� number, measurement, space, chance and data understandings (primarily for system monitoring purposes) cannot provide the same fine-grained information about students� understandings of fractions, as a tool designed specifically for that purpose. In their practice, accomplished teachers use diagnostic information from tools of different diagnostic power as they �drill down� to obtain detailed information about a student�s knowledge, skills and understandings. What does drill down assessment look like? It exposes how students are thinking about a problem as well as what they know.

10. For example, This task comes from a set of materials originally used in a Victorian study to explore students� science beliefs Students were presented with a serious of cartoons, and open ended questions designed to elicit conceptual understandings and misunderstandingsFor example, This task comes from a set of materials originally used in a Victorian study to explore students� science beliefs Students were presented with a serious of cartoons, and open ended questions designed to elicit conceptual understandings and misunderstandings

11. 3 Indicates that resistance forces are acting on the skateboard and they cause it to decelerate 2 Suggests that forces such as gravity and wind resistance will cause it to stop 1 Suggests that the force is in the skateboard and gets used up Links force and motion (eg because there is no kicking there will be no motion) The marking guide for answers to this question addresses three levels of understanding. At the lowest level�. At the highest level� The marking guide for answers to this question addresses three levels of understanding. At the lowest level�. At the highest level�

12. Learning is enhanced when teachers identify and work from individuals� current knowledge, skills and beliefs. Bransford, Brown and Cocking (2000) Informative assessment of this kind enables teachers to identify and in their teaching practice work from students� current knowledge skills and beliefs rather than from their expectation of what students should know based in particular on their age or Year levelInformative assessment of this kind enables teachers to identify and in their teaching practice work from students� current knowledge skills and beliefs rather than from their expectation of what students should know based in particular on their age or Year level

13. The second body of research that informative assessment draws on can be summarised as the finding that learning is an ongoing process. Throughout the years of school and life beyond school individuals build on what they know. They refine and deepen that knowledge and make connections to broaden the reach of their understandings. What implications do these findings have for the practice of informative assessment? The second body of research that informative assessment draws on can be summarised as the finding that learning is an ongoing process. Throughout the years of school and life beyond school individuals build on what they know. They refine and deepen that knowledge and make connections to broaden the reach of their understandings. What implications do these findings have for the practice of informative assessment?

14.

15. Monitoring the ongoing process of learning is facilitated by the use of a map of learning against which learning can be assessed, monitored and reported. These frameworks � sometimes called progress maps or learning progressing provide a reference against which learning can be assessed, monitored and reportedThese frameworks � sometimes called progress maps or learning progressing provide a reference against which learning can be assessed, monitored and reported

16. deal with difficult texts and to complete sophisticated reading tasks. They can deal with information that is difficult to find in unfamiliar texts, especially in the presence of closely competing information, show detailed understanding of these texts and sort out which information is relevant to the task. They are able to evaluate texts critically, draw on specialized knowledge to build hypotheses, and cope with concepts that may be contrary to expectations. cope with difficult tasks, such as locating embedded information, construing meaning of part of a text through considering the text as a whole, and dealing with ambiguities and negatively worded ideas. They show accurate understanding of complex texts and are able to evaluate texts critically. deal with moderately complex reading tasks, such as finding several pieces of relevant information and sorting out detailed competing information requiring consideration of many criteria to compare, contrast or categorise. They are able to make links between different parts of a text and to understand text in a detailed way in relation to everyday knowledge. cope with basic reading tasks, such as locating straightforward information, making low-level inferences, using some outside knowledge to help understand a well defined part of a text, and applying their own experience and attitudes to help explain a feature of a text. deal with only the least complex reading tasks developed for PISA, such as finding explicitly stated pieces of information and recognising the main theme or author�s purpose in a text on a familiar topic when the required information is readily accessible in the text. They are also able to make a connection between common, everyday knowledge and information in the text. This framework based on the Program for International Student Assessment (PISA) reading provides an example. Maps of this kind are underpinned by the intention to describe learning, in this case reading, from low level to sophisticated understanding along a single continuum. For ease of monitoring and reporting each continuum is divided into levels and a description developed to illustrate in words typical achievement at each level along the continuumThis framework based on the Program for International Student Assessment (PISA) reading provides an example. Maps of this kind are underpinned by the intention to describe learning, in this case reading, from low level to sophisticated understanding along a single continuum. For ease of monitoring and reporting each continuum is divided into levels and a description developed to illustrate in words typical achievement at each level along the continuum

17. Most importantly, we are not talking just about generalisations of levels of achievement and growth but about what growth actually looks like: Accompanying work samples, provide concrete references for teachers parents and students as basis for discussion about what is valued, the features of quality work, and ultimately what it means to progress in an area of learning. Here are some examples from Australian students� writing. The work illustrates early children�s writing While you look at the next couple you might think about how a reference of this kind can assist students to monitor their own learning. Some years ago I met with a researcher in Seattle who talked about a school where teachers put progress maps and accompanying samples of student work along the walls of the classroom and students were coming to the teacher saying for example, my work is now at level 3 not level 2 and the teacher would say, this piece looks like your work is at level 3 but show me several work samples Most importantly, we are not talking just about generalisations of levels of achievement and growth but about what growth actually looks like: Accompanying work samples, provide concrete references for teachers parents and students as basis for discussion about what is valued, the features of quality work, and ultimately what it means to progress in an area of learning. Here are some examples from Australian students� writing. The work illustrates early children�s writing While you look at the next couple you might think about how a reference of this kind can assist students to monitor their own learning. Some years ago I met with a researcher in Seattle who talked about a school where teachers put progress maps and accompanying samples of student work along the walls of the classroom and students were coming to the teacher saying for example, my work is now at level 3 not level 2 and the teacher would say, this piece looks like your work is at level 3 but show me several work samples

20. Research suggests that �. Including classroom teachers, curric. Coordinators, principals�. It is easy to assume that this understanding of growth is their already and does not require any effort shared Golf ball storyResearch suggests that �. Including classroom teachers, curric. Coordinators, principals�. It is easy to assume that this understanding of growth is their already and does not require any effort shared Golf ball story

21.

22. What does this coordinated and shared process look like in practice? Here I want to frame my answer to this question around 3 of the steps as shown in this picture � a variation of many such pictures you will all have seen in curriculum and assessment documents. What does this coordinated and shared process look like in practice? Here I want to frame my answer to this question around 3 of the steps as shown in this picture � a variation of many such pictures you will all have seen in curriculum and assessment documents.

23. My focus will be on the 3 of the 4 steps in the cycle: Orienting and preparing to take a whole school approach to informative assessment; collecting and analysing evidence in order to take action that is both focused on the needs of each individual students within a coherent whole school strategy And taking action on the basis of evidence � again with a beyond year level only focus. My focus will be on the 3 of the 4 steps in the cycle: Orienting and preparing to take a whole school approach to informative assessment; collecting and analysing evidence in order to take action that is both focused on the needs of each individual students within a coherent whole school strategy And taking action on the basis of evidence � again with a beyond year level only focus.

24. Let me begin with orienting and preparing. What does this mean. I suggest, in the context of a whole school approach to informative assessment it means� And this commitment to the systematic collection of evidence as a basis for improving learning is characterised in practice by a number of features Let me begin with orienting and preparing. What does this mean. I suggest, in the context of a whole school approach to informative assessment it means� And this commitment to the systematic collection of evidence as a basis for improving learning is characterised in practice by a number of features

25. Building a culture which forms the social context for an evidence based culture of intervention to improve learning Developing a whole school plan which forms the structural backbone to the collection of achievement information This plan specifies the kinds of data �of different diagnostic power) that will be collected (including for example, NAPLAN, perhaps commercially developed tests, and teacher developed assessments) the time of year that the data will be collected and who will collect the data. Providing ongoing prof learning opportunities that ensure teachers have the skills and understandings to collect and use assessment information to improve learning And attending to, indeed requiring openness and collaboration. Note response to argument that have no time: Collaborate, consolidate; attend to efficienciesBuilding a culture which forms the social context for an evidence based culture of intervention to improve learning Developing a whole school plan which forms the structural backbone to the collection of achievement information This plan specifies the kinds of data �of different diagnostic power) that will be collected (including for example, NAPLAN, perhaps commercially developed tests, and teacher developed assessments) the time of year that the data will be collected and who will collect the data. Providing ongoing prof learning opportunities that ensure teachers have the skills and understandings to collect and use assessment information to improve learning And attending to, indeed requiring openness and collaboration. Note response to argument that have no time: Collaborate, consolidate; attend to efficiencies

26. An example Ballarat and Clarendon -- an expectation/requirement of openness and participation in ongoing professional learning Ballarat and Clarendon an independent early learning to Year 12 co-ed school in regional Victoria. When I first visited B&C a few years ago I was taken to see their central room �. One way mirrors Camera to watch and home in on� Classes cycle through Videotape used for staff room discussion Many teachers are uncomfortable at sometimes horrified at this prospect. Yet we expect students every day of their school lives to expose their strengths and weakness to their peers When I spoke about B&C a few weeks ago at a workshop, several teachers from B&C were there and they provided an update: Now there are 7 rooms, never know when video on, quite natural to them, and need to teach in this room as part of job interview Turn to your neighbour and give a thumbs up or thumbs down. Ballarat and Clarendon an independent early learning to Year 12 co-ed school in regional Victoria. When I first visited B&C a few years ago I was taken to see their central room �. One way mirrors Camera to watch and home in on� Classes cycle through Videotape used for staff room discussion Many teachers are uncomfortable at sometimes horrified at this prospect. Yet we expect students every day of their school lives to expose their strengths and weakness to their peers When I spoke about B&C a few weeks ago at a workshop, several teachers from B&C were there and they provided an update: Now there are 7 rooms, never know when video on, quite natural to them, and need to teach in this room as part of job interview Turn to your neighbour and give a thumbs up or thumbs down.

27. Next step ..Next step ..

29. It looks like 3 steps often confused sometimes given insufficient attention Are there patterns in students� responses? Do incorrect responses indicate common misconceptions of gaps in students� learning? (Note distractor analysis) Are there strengths/weaknesses in relation to particular concepts and/ or skills? References for interpreting (interrogating) Have students performed as the teacher expected? NAPLAN Have students performed to age/grade expectations? Possible explanations (theorising) off day? (the individual student) difficulties for particular students (sub groups) class assessments not addressing same curriculum expectations? (the teacher) some teaching approaches more effective than others? (the school) appropriateness of curriculum emphasis? (the system) It looks like 3 steps often confused sometimes given insufficient attention Are there patterns in students� responses? Do incorrect responses indicate common misconceptions of gaps in students� learning? (Note distractor analysis) Are there strengths/weaknesses in relation to particular concepts and/ or skills? References for interpreting (interrogating) Have students performed as the teacher expected? NAPLAN Have students performed to age/grade expectations? Possible explanations (theorising) off day? (the individual student) difficulties for particular students (sub groups) class assessments not addressing same curriculum expectations? (the teacher) some teaching approaches more effective than others? (the school) appropriateness of curriculum emphasis? (the system)

33. An example Our Lady of the Sacred Heart --and expectation/ requirement to teach, reflect and use assessment evidence collaboratively Coed primary in low socio ec area of inner Sydney guided reading program talk read and think way purposefully thru texts P-3 classes divided into 6 groups of 11 student each group class teachers, 2 reading support teachers, an English teacher and Esl teacher and the principal Regular assessment -- every few weeks to regrouping after a planning meeting of all teachers with focus on at-risk students Outstanding results. Many possible explanations�Turn to the person next to you and in one sentence say why you think this school was so successfulCoed primary in low socio ec area of inner Sydney guided reading program talk read and think way purposefully thru texts P-3 classes divided into 6 groups of 11 student each group class teachers, 2 reading support teachers, an English teacher and Esl teacher and the principal Regular assessment -- every few weeks to regrouping after a planning meeting of all teachers with focus on at-risk students Outstanding results. Many possible explanations�Turn to the person next to you and in one sentence say why you think this school was so successful

34. What implications do these findings have for the practice of informative assessment? What implications do these findings have for the practice of informative assessment?

36. Delightful example of deep learning This almost by way of aside is a cartoon that appeared in one of the early newspaper reports on PISA For me it�s a wonderful example of the misconceptions that many young students hold. I think it would make a wonderful piece of stimulus for materials that tap into students mathematical beliefs.Delightful example of deep learning This almost by way of aside is a cartoon that appeared in one of the early newspaper reports on PISA For me it�s a wonderful example of the misconceptions that many young students hold. I think it would make a wonderful piece of stimulus for materials that tap into students mathematical beliefs.

37. 1. Assessment should reveal the depth of individuals� understandings. assessment as exploration

38. Lovely Mosquito

39. Does the writer think the mosquito is lovely?Explain your answer.

40.

41. 

42. Why did the lion ask the animals to visit him? 2 points Recognises the intent of the lion to trick and/or eat the animals 1 point Gives a literal answer. (He was ill. He was sick.)

43. 3. Assessment may need to draw on different and sophisticated assessment approaches. An understanding of methods of assessment The promise of new technologiesAn understanding of methods of assessment The promise of new technologies

44. From international study in electronic readingFrom international study in electronic reading

49.

50. Dweck in 1989 focused on defining the positive classroom climate or culture for learningDweck in 1989 focused on defining the positive classroom climate or culture for learning

51. Dylan Wiliam � homed in on the feedback we provide students If students are provided with a score or a grade on an individual piece of work, they will attend to that, even if they are provided with descriptive feedback as well. The research suggests that if we want students to attend to the feedback teachers provide, the feedback should be in the form of written comments and not a score or grade. �If the feedback is to be effective, it must be focused on what the individual student needs to do to improve (ie it must be task-involving) rather than focusing attention on to the learner and her or his self-esteem (ie ego-involving)� (Wiliam, 1998. p. 6). Dylan Wiliam � homed in on the feedback we provide students If students are provided with a score or a grade on an individual piece of work, they will attend to that, even if they are provided with descriptive feedback as well. The research suggests that if we want students to attend to the feedback teachers provide, the feedback should be in the form of written comments and not a score or grade. �If the feedback is to be effective, it must be focused on what the individual student needs to do to improve (ie it must be task-involving) rather than focusing attention on to the learner and her or his self-esteem (ie ego-involving)� (Wiliam, 1998. p. 6).

52.

53. Two examples Seattle Toronto

54. Learning is enhanced by the ability to monitor one�s own learning. Bransford, Brown and Cocking (2000)