2018 – 19 CAASPP The Results Are In … Now What?

1. 2018–19 CAASPP The Results Are In … Now What?

2. Overall Workshop Objectives (1) • Use a data analysis protocol that enables a root cause examination of past and current programs, policies, and practices, leading to applicable and appropriate actions to improve teaching and learning • Analyze and use the 2018–19 (and previous years’) results to inform teaching and learning The Results Are In…Now What?

3. Overall Workshop Objectives (2) • Access and use supplementary resources and reference materials to inform the interpretation of student results • Consider ways to display results for varied stakeholder audiences for a particular purpose The Results Are In…Now What?

4. Norms • Be present and engaged • Respect the opinions of others • Use Padlet as a digital parking lot for questions • LINK: http://bit.ly/CAASPPResults19P • Others? The Results Are In…Now What?

5. Welcome and Introduction Module 1

6. Learning Goals for Module 1 • Participants will understand what the day’s learning entails. • Participants will identify local educational agency (LEA) or school goals, focus, and target areas as a context for the day. The Results Are In…Now What?

7. Success Criteria for Module 1 • Participants will identify questions pertaining to their own schools and LEAs, which can be addressed through summative data analysis. The Results Are In…Now What?

8. As You Engage, Be Thinking … • How might you use this information at your LEA? • What knowledge exists already? • What can be adopted or adapted? The Results Are In…Now What?

9. Data Analysis Processes Plan/Do/Study/Act The Four Rs CAASPP Institute The Results Are In…Now What?

10. Framing the Conversation The Results Are In…Now What?

11. Example: Manteca Unified • Engaged all administrators and instructional leaders in professional learning series, “Using Assessment to Shift Classroom Instruction” • Used 4Rs (research, recall, reflect, respond) protocol to analyze summative and local assessment data • Identified mathematics and mathematics practices as a focus area • Deepened understanding of implications for teaching and learning through use of Building Educator Assessment Literacy (BEAL) The Results Are In…Now What?

12. Example: Brea Olinda Unified • Reformatted the handout • Tailored Guiding Questions to local context • First year: principals as leads • Second year: LEA-wide grade-level teams with Teachers on Special Assignment as leads • Third year: deeper dive into resources The Results Are In…Now What?

13. Reflect on Your LEA’s Data Analysis Practices Discuss the following with your tablemates and be prepared to share with the larger group. • What current practices are in place to analyze summative assessment data at your site? • Who is involved? The Results Are In…Now What?

14. Using Data to Answer Questions • Choose a focus based on an LEA goal, priority, or planned improvement for this year. It could be a grade band, subgroup, content area goal, or initiative.  • What questions do you want the data to help you answer? • How does this focus connect to your LEA’s expectations in regard to school improvement and student achievement? The Results Are In…Now What?

15. Revisiting Learning Goals for Module 1 • Participants will understand what the day’s learning entails. • Participants will identify LEA or school goals, focus, and target areas as a context for the day. The Results Are In…Now What?

16. Educational Assessment Module 2

17. Learning Goals for Module 2 (1) • Understand different types of assessments and the kinds of evidence elicited from students for each • Understand the purposes of different types of assessment within a balanced assessment system The Results Are In…Now What?

18. Learning Goals for Module 2 (2) • Understand that different types of assessments result in different kinds of data • Understand how evidence-centered design informs the development of assessments that are directly connected to the learning standards The Results Are In…Now What?

19. Success Criteria for Module 2 • Match different sources of assessment data with different stakeholder needs • Identify different types of assessment in your LEA and the kinds of evidence of student learning they elicit The Results Are In…Now What?

20. Comprehensive Assessment System The Results Are In…Now What?

21. Assessment Literacy (1) • Assessment literacy is about knowing how to assess student knowledge, skills, and abilities, and using the results to improve student learning and program effectiveness. • Though the focus of this workshop is summative assessment results, keep in mind the context of a comprehensive, balanced, assessment system. The Results Are In…Now What?

22. Assessment Literacy (2) • Assessment types can be differentiated by a number of factors: • Scale or “grain size” • Frequency & timing • Purpose • Classroom use • Test conditions • Flexibility • Design • Use of resulting data The Results Are In…Now What?

23. Assessment Types The Results Are In…Now What?

24. Activity: Assessment Inventory • Locate Handout 2.2: Assessment Inventory. • Choose a context that has to do with the LEA goal identified in Module 1. • Complete the inventory of assessments used in that context, and the details about them. The Results Are In…Now What?

25. Local Assessment Reflection • Reflect on how the different assessments identified in your assessment inventory communicate evidence about student learning to different stakeholder groups. • Discuss with an elbow partner. The Results Are In…Now What?

26. Evidence-Centered Design Evidence-centered design (ECD) ensures that each item or task elicits evidence about the target of an assessment that can be used to support a claim about a student’s development of the knowledge, skill, or ability contained in a content standard. The Results Are In…Now What?

27. Claims • Broad statements of the assessment system’s learning outcomes: • English language arts (ELA) reading claim: (I claim that …) “Students can read closely and analytically to comprehend a range of increasingly complex literary and informational texts.” • If I notice the following evidence … as spelled out in targets. The Results Are In…Now What?

28. Targets • Statements that describe the evidence needed to back up a claim • Grade 5 Example • Reading, Literary Texts • Target 7, Language UseDetermine the meaning of words and phrases including figurative language (e.g., metaphors and similes) or demonstrate understanding of nuances in word meanings in context. For more information, refer to Connection to Items slide. The Results Are In…Now What?

29. Conceptual Framework The Results Are In…Now What?

30. Summative Data • Achievement • Claims • Targets The Results Are In…Now What?

31. Connection to Items Reading, Literary Text Target 7. LANGUAGE USE (example is Grade 5) Determine the meaning of words and phrases including figurative language (e.g., metaphors, similes) or demonstrate understanding of nuances in word meanings in context. The Results Are In…Now What?

32. Required Evidence Item specifications break down the types of evidence needed to indicate that a student possesses the knowledge, skills, and abilities required for each target. Smarter Balanced item specifications can be found here: https://www.smarterbalanced.org/assessments/development/#item The Results Are In…Now What?

33. Rigor • What verbs do you notice in the assessment target and standards? • Target 7 is Depth of Knowledge 2 and 3. The Results Are In…Now What?

34. Revisiting Learning Goals for Module 2 • Understand different types of assessments and the kinds of evidence elicited from students for each • Understand the purposes of different types of assessments within a balanced assessment system • Understand that different types of assessments result in different kinds of data • Understand how evidence-centered design informs the development of assessments that are directly connected to the learning standards The Results Are In…Now What?

35. Scores and Sources Connecting to Teaching and Learning Module 3

36. Learning Goals for Module 3 (1) • Understand key reporting categories, how they are used, and why they are important • Understand information on the public state reporting site and the Online Reporting System (ORS) The Results Are In…Now What?

37. Learning Goals for Module 3 (2) • Know where to find related resources • Understand how to use related resources to go deeper into scores and reporting The Results Are In…Now What?

38. Success Criteria for Module 3 • Locate and download score reports and sources of information that are related to reporting • Discuss with others what different scores and reports mean • Identify how score reports and resources can be used in schools and LEAs The Results Are In…Now What?

39. LEA and School Goals LEA Goals–Expected Outcomes (Examples) 1. Increase the number of our grade five students proficient in CAASPP ELA from 52% in 2019 to 65% in 2020. 2. To address our existing achievement gap, increase the number of English Learners proficient in CAASPP mathematics from 45% in 2019 to 55% in 2020. The Results Are In…Now What?

40. Remember … This Is Where Change Happens The assessment system is designed to support teachers in facilitating student mastery of state standards. ​ The Results Are In…Now What?

41. What Do Scores Tell Us? Module 3A

42. Smarter Balanced Reporting Categories • Achievement levels and scale scores • Level 4—Standard Exceeded • Level 3—Standard Met • Level 2—Standard Nearly Met • Level 1—Standard Not Met • Claim performance levels • Above, Near, Below Standard • Student Reporting Groups • Disability, Economic, Language, Ethnicity, Gender The Results Are In…Now What?

43. Scale Score Ranges—Example • Students may increase their scale score from year to year but drop achievement levels • For a full list of English language arts/literacy (ELA) and mathematics scale score ranges, refer to Handout 3.1. The Results Are In…Now What?

44. Smarter Balanced Threshold Scale Scores ELA Mathematics The Results Are In…Now What?

45. Achievement Level Descriptors (ALDs) ALDs describe performance on a standardized test in terms of levels or categories of performance.  The Results Are In…Now What?

46. Claim Achievement Level Descriptors • Locate Handout 3.2. • Consider the language that describes student performance across each claim. • With a neighbor, discuss what the information contained in “Claim Achievement Level Descriptors” can tell you about student performance and how it could be used to inform teaching and learning practices. The Results Are In…Now What?

47. Public Reporting Website Module 3B

48. Public Reporting Site (1) • Assessments  • Smarter Balanced Summative Assessments for ELA and mathematics • California Alternate Assessments (CAA) • California Spanish Assessment (CSA) The Results Are In…Now What?

49. Public Reporting Site (2) • Smarter Balanced ELA/literacy and mathematics Achievement results in percentages • LEA, school, student group • Proficiency by level • Claim data by performance level • ALDs and scale score range reference documents • Historical data • Summary, Change Over Time • Comparison tool The Results Are In…Now What?

50. Public Reporting Site (1) • Assessments  • Smarter Balanced Summative Assessments for ELA and mathematics • California Alternate Assessments (CAA) • California Spanish Assessment (CSA) The Results Are In…Now What?