GLCE and YOU! Focus on Assessment

1. GLCE and YOU!Focus on Assessment January 12 and 13, 2005

2. Assessment Crisis Assessment for Learning Assessment of Learning GLCE and YOU-Focus on Assessment

3. Crucial Distinction Assessment OF Learning: How much have students learned as of a particular point in time? Assessment FOR Learning: How can we use assessment to help students learn more? GLCE and YOU-Focus on Assessment

4. Key Questions Are our current approaches to assessment • improving student learning? • helping teachers improve their craft? GLCE and YOU-Focus on Assessment

5. A comparative view GLCE and YOU-Focus on Assessment

6. Assessment FOR learning • Provides timely information that helps teachers teach and students learn • Is currently the weak link in our quest for continuous improvement and our effort to “leave no child behind” GLCE and YOU-Focus on Assessment

7. When Assessing for learning: Teachers… • understand and articulate achievement targets before teaching. • inform students of targets in student friendly terms. • create assessments that accurately reflect student achievement. • use assessments positively to build student confidence in themselves as learners • provide frequent, descriptive feedback to students to help them improve. • continuously adjust instruction based on classroom assessment results. GLCE and YOU-Focus on Assessment

8. What we really need to do… • Plan carefully for assessment use. • Understand what information we need from assessment. • Determine if an assessment can provide the information required for the intended purpose. GLCE and YOU-Focus on Assessment

9. Research demonstrates clearly… • Strengthening formative assessment practice results in significant, substantial learning gains. • Improved formative assessment helps the lowest achieversmost. • The common feature of studies showing large effect sizes accurate, high quality formative assessments. GLCE and YOU-Focus on Assessment

10. 1. Does better formative assessment lead to higher test scores?2. Does formative assessment need improving?3. If so, what improvement is needed? Black & William Questions: GLCE and YOU-Focus on Assessment

11. What They Found • Yes - better formativeassessmentdid lead to higher test scores. • Yes – the development and use of formative assessments greatly need improvement. • Accuracy, descriptive feedback, student involvement are areas most in need of improvement. GLCE and YOU-Focus on Assessment

12. Research - Effects of Major Initiatives Study S.D. Gains Bloom (1984) 1.0 to 2.0 * Black and Wiliam (1998) .5 to 1.0** Meisels, et al. (2003) .7 to 1.5 Rodriguez (2004) .5 to 1.8** * Rivals one-on-one tutorial instruction ** Largest gains for low achievers GLCE and YOU-Focus on Assessment

13. 1.0 Standard Deviation equals: • 30+ Percentile Points On ITBS (middle of score range) • 3-4 Grade Equivalents • 100 SAT Score Points • 5-6 ACT Score Points • U.S. TIMMS Rank From 23rd to Top 5 • Potential Elimination of Score Gaps Unprecedented Achievement Gains GLCE and YOU-Focus on Assessment

14. Largest effects when teachers… • increased the accuracy of formative assessments • enhanced the quality of formative assessments • increased descriptive feedback • increased student involvement GLCE and YOU-Focus on Assessment

15. Think-Pair-Share Think of a time when you’ve used formative assessment really well in your classroom. • How did this effect your teaching? • How did this impact student learning? GLCE and YOU-Focus on Assessment

16. Standards Of Assessment Quality Standard 1 Assess what? Are targets clear and appropriate? Standard 2 Why assess? Are users and uses clear? Standard 3 Assess how? Is the assessment method appropriate? Standard 4 How much evidence? Is achievement adequately sampled? Standard 5 Assess accurately? Has bias been minimized?

17. Standard 1: Clear and Appropriate Learning Targets • What’s the Target? A learning target is any achievement expectation we hold for students. It’s a statement of what we want the students to know and be able to do. GLCE and YOU-Focus on Assessment

18. Kinds of Achievement Targets Master Factual and Procedural Knowledge Some to be learned outright Some to be retrieved using reference materials Use Knowledge to Reason and Solve Problems Analytical or comparative reasoning Synthesizing Classifying Induction and deduction Critical/evaluative thinking GLCE and YOU-Focus on Assessment

19. Kinds of Achievement Targets, cont. Demonstrate Mastery of Specific Skills Speaking a second language Giving an oral presentation Working effectively on a team Science process skills Create Quality Products Writing samples Artistic products Research reports Shop projects Science exhibits GLCE and YOU-Focus on Assessment

20. Kinds of Achievement Targets, cont. Acquire Positive Affect/Dispositions Positive self-concept Desire to learn/read/think critically Positive attitude toward school Good citizenship Respect toward self and others Flexibility Perseverance GLCE and YOU-Focus on Assessment

21. Standard 1: Clear and Appropriate Learning Targets Is This a Target? • Senior project • Bird feeder • Use a band saw safely • State report • Diorama GLCE and YOU-Focus on Assessment

22. Standard 1: Clear and Appropriate Learning Targets A Mathematics Example • Math • Decimals • Page 152 in the book • Going on a “decimal hunt” • Read decimals and put them in order GLCE and YOU-Focus on Assessment

23. What’s the Target? • Examples of learning targets taken from a 4th grade social studies text: • Knowledge Targets (recall) • Name at least two regions of which Washington is a part. • Tell at least one way that location has affected Washington's history. • Name five main parts of Washington State and describe key land and water forms of each. • Define the term natural resources and give examples of those found in Washington. • Examples of key words: tell, understand, name, describe, list, identify, give examples. • Knowledge Targets (procedural) • How to use map scales to measure distance- • How to use latitude and longitude to locate places on a map or globe. • Examples of key words: how to (followed by some procedure). GLCE and YOU-Focus on Assessment

24. What’s the Target? continued… Examples of learning targets taken from a 4th grade social studies text Reasoning Target Give examples of differences between coastal and plateau cultures and relate these to differences in the natural environment. Examples of key words: classify, compare, contrast, analyze, synthesize, determine, evaluate, justify, construct support, draw conclusions. Skill Target Use map scales to measure distance. Examples of key words: read, speak, assemble, operate, use, demonstrate. Product Target Make a relief map of any region of the whole state, or make maps of products, points of interest, or land uses. Examples of key words: create, design, make, produce, write, draw. GLCE and YOU-Focus on Assessment

25. Fast-Tracked Targets are stated, selective, and easy to find Targets are important – worth the assessment time devoted to them. Targets are related clearly to district/state standards/outcomes. There is an effort to define targets: examples of student work, references to definitions, references to performance criteria, and/or a table of specifications; it is clear that the author understands that such references help users define the targets. The targets are clear enough that educators would more or less interpret them the same. Target descriptions and definitions reflect an understanding of best thinking in the field. There is an appropriate mix of targets and/or there is evidence of long-term thinking – how the targets in the current assessment fit with the plan for the year. Standard 1 – Features of Clear & Appropriate Student Learning Targets GLCE and YOU-Focus on Assessment

26. On Track, But Needs Work Targets are listed, but they might be stated differently in different places, scattered, or require some work to find. Some of the targets are essential, but there also seems to be some dead wood that might profitably be cut; some targets seem to have been chosen because they were easy to assess. Targets seem to be retrofitted to an already existing test; as a result, one might feel somewhat dissatisfied that the assessment is not as well thought out, comprehensive, or as focused as it might have been had the targets been identified first. Although targets are stated, there is some question as to their meaning-different educators might define the targets differently At first glance there appears to be a connection between stated targets and local content standards, but on closer examination the connection is not clear. Although the author provides local content standards, rubrics, etc., it is not clear that he or she knows that these help users understand the nature of the targets being assessed. Rubrics only partially help define the targets. Some of the targets represent the best thinking in the field; others do not. Standard 1 – Features of Clear & Appropriate Student Learning Targets GLCE and YOU-Focus on Assessment

27. Side-Tracked Stated targets are broad, general and vague; there is little attempt at clarification No targets are stated. There is little focus; everything is listed. Statements of targets ramble; the author lists one and later seems to list others; targets have to be inferred from the assessment itself. Targets are stated, but seem trivial; why spend time assessing this? The description of targets doesn’t reflect an understanding of best thinking in the field. There is a poor mix of targets; the author might, for example, have chosen only the easiest targets to asses; or, there is little evidence of long-term thinking how the targets in the current assessment fit into the overall plan for the year. There is no connection made to district and state standards or outcomes. Targets and tasks are mixed up. Standard 1 – Features of Clear & Appropriate Student Learning Targets GLCE and YOU-Focus on Assessment

28. Standard 2: Clear and Appropriate Users and Uses Who will use assessment results and for what purpose? • Classroom- students, teachers, parents • InstructionalSupport - curriculumspecialists, principals, counselors • Policy– superintendent, school board, state department of education, legislators, taxpayers GLCE and YOU-Focus on Assessment

29. GLCE and YOU-Focus on Assessment

30. Think-Pair-Share Think of a time when you’ve used formative assessment data effectively. • What did you do that made your use of the data effective? • How did this effect your teaching? • How did this impact student learning? GLCE and YOU-Focus on Assessment

31. Fast-Tracked It is clear who the intended users & uses are Users and uses are appropriate and focused. There are statements relating assessment design to users and uses. It is clear how the assessment can be used to inform instruction. Standard 2:Clear and Appropriate Users and Uses GLCE and YOU-Focus on Assessment

32. On Track, But Needs Work Author has considered several users/uses, but does not clearly determine how assessment design should differ for these audiences. Users and uses are implied but not clearly stated.. Implications for instructional decision making are there, but must be inferred. Users and uses are stated, but there is a question of appropriateness. Users and uses are stated, but the author doesn’t seem to understand the importance of stating them. Standard 2: Clear and Appropriate Users and Uses GLCE and YOU-Focus on Assessment

33. Side-Tracked There are too many users and uses; cannot satisfy all stated purposes in single assessment. No purposes are stated; it is not clear why assessment is being given. The stated purpose doesn’t match the assessment. The only purpose, ever, is grading. The author doesn’t appear to be aware that assessments should be designed with users and uses in mind. It is not clear how results would inform future instruction. Standard 2: Clear and Appropriate Users and Uses GLCE and YOU-Focus on Assessment

34. Standard 3: Target-Method-Match Determining most efficient and effective way to assess targets: • Identify critical targets • Meet purpose of instruction • Review possible methods • Identify rationale for methods selected GLCE and YOU-Focus on Assessment

35. Possible Assessment Methods Selected Response/Short Answer ○ Multiple Choice ○ True/False ○ Matching ○ Fill in the blank ○ Label a diagram ○ Sentence Essay Assessments – extended written response Performance Assessments ○ Demonstrating skills ○ Developing a product Personal Oral Communication ○ Question and Answer ○Conferences ○ Interviews ○ Oral Examinations GLCE and YOU-Focus on Assessment

36. Target-Method Match Activity GLCE and YOU-Focus on Assessment

37. Fast-Tracked The assessment method matches purpose and target. There’s a reasonable rationale for the method used. An assessment outline clearly shows how each target is to be measured and its relative importance. There is a balanced and appropriate mix of assessment methods. The author carefully chose when and how to use performance assessments; using simpler methods when possible. There is a good match between instructional emphasis on targets and assessment emphasis on targets. Standard 3 – Features of Target-Method Match GLCE and YOU-Focus on Assessment

38. On Track, But Needs Work Author has used a variety of assessment methods, but it is somewhat unclear why. Some of the assessment methods could be improved. Author overuses performance assessment when a simpler method might be cleverly applied. Target/method matching are not clearly explained and must be inferred. There are some mismatches among targets, instruction and/or assessment methods. Standard 3 – Features of Target-Method Match GLCE and YOU-Focus on Assessment

39. Side-Tracked The assessment method is inappropriate for the learning target. Only one type of assessment is used for every target. There is no apparent rationale for the assessment methods used. Over-reliance on assessing only the higher-level skills, without consideration for assessing prerequisite skills which might require a simpler assessment method. Mismatch between the target being assessed and the targets emphasized in instruction. Many missed opportunities for assessment. Standard 3 – Features of Target-Method Match GLCE and YOU-Focus on Assessment

40. Standard 4: Sampling - The Basics • Any assessment is a sample of the possible items/questions/tasks a teacher could use to assess a target. • To determine how much is enough, ask: “Does my assessment cover all important aspects of the target and give students enough chance to show what they know?” GLCE and YOU-Focus on Assessment

41. Standard 4: Sampling - The Basics • Example 1: How many landings, in which types of conditions, would it take for you to certify a pilot’s competence? • Example 2: How many writing samples, of which types, would it take for you to certify that a student can write well for different audiences and purposes? GLCE and YOU-Focus on Assessment

42. Standard 4: Sampling - The Basics We have sampled enough when… we are confident we can predict how student will do on the next item/test/task based on previous responses to items/tests/tasks of the same target. GLCE and YOU-Focus on Assessment

43. Fast-Tracked The author has defined the target/domain from which the specific assessment tasks have been sampled. The sample of student performance will accomplish the purpose. There are enough samples of student work to get a good estimate of achievement. There are not too many tasks, nor too few. The tasks cover the learning target/domain well. The sample matches the breadth of the target and/or the importance of results. Standard 4– Features of Sampling GLCE and YOU-Focus on Assessment

44. On Track, But Needs Work Author seems to have covered the learning targets/domain well, but has not made a clear enough description of domain to know for certain. The author has students doing more tasks than necessary to get a good estimate of achievement. There is good coverage of the domain of skills needed to make a stable estimate of achievement, but assessment would benefit from a few additional tasks. Although sampling might be ok for some uses, the stakes are such that additional samples would be beneficial. Sampling seems acceptable, but is not explicitly addressed and must be inferred. Some outcomes are sampled well, some are not. Standard 4– Features of Sampling GLCE and YOU-Focus on Assessment

45. Side-Tracked There are not enough tasks to draw the desired conclusion. The tasks do not cover the ground well. The author doesn’t seem to be aware that all assessments sample from a domain and that the domain must be defined. The sample doesn’t match the breadth of the target nor the importance of the results. Standard 4– Features of Sampling GLCE and YOU-Focus on Assessment

46. Standard 5: Avoid sources of bias and distortion Bias – A test or item is said to be biased when a certain students or groups have an unfair advantage over others. Distortion- Biased tests or items cause results to inaccurately reflect student ability. GLCE and YOU-Focus on Assessment

47. Brainstorm potential sources of bias - • Test format/content • Testing environment • Student input GLCE and YOU-Focus on Assessment

48. Standard 5: Avoid sources of bias and distortion Barriers common to all assessment methods: Occur within the student: • Language barriers • Emotional upset • Poor health • Physical handicap • Peer pressure to mislead assessor • Lack of motivation at time of assessment • Lack of understanding how to take tests • Lack of personal confidence leading to evaluation anxiety GLCE and YOU-Focus on Assessment

49. Standard 5: Avoid sources of bias and distortion Occur within the assessment context: • Noise distractions • Poor lighting • Discomfort • Lack of rapport with assessor • Cultural insensitivity in assessor or assessment • Lack of proper equipment GLCE and YOU-Focus on Assessment

50. Standard 5: Avoid sources of bias and distortion Arise from the assessment itself (regardless of method): • Directions lacking or vague • Poorly worded questions • Poor reproduction of test questions • Missing information GLCE and YOU-Focus on Assessment