Oregon Rubric & Quality Review Training Session: Mathematics

1. Oregon Rubric & Quality Review Training Session: Mathematics Pilot Review Process

2. What is a Pilot Review? • Oregon has specific laws that outlines how state reviews are carried out and when (ORS 337 & OAR 581-11). • 30+ year old laws (Basal focused, seven year review cycle) • Review is sustained by publisher fees • State math review has been moved from 2014 to 2016 • Will work in the 2015 session to update the state review process • Districts are able to independently review and adopt at any time, provided they use the board approved criteria (OAR 581-022-1622) • Pilot process to support the local review of materials • Summer 2014: SOESD and Hillsboro “Pilot” review process for math • Fall 2014: share results and supporting documents • Summer 2015: aim to support more regional reviews of math

3. Session Goals Use the Oregon quality review process (based on EQuIP and IMET) to determine the quality and alignment of lessons and units to the Common Core State Standards (CCSS) in mathematics During this session, reviewers will: • Develop their abilities to use Oregon math materials criteria to provide observations about CCSS-aligned instructional materials and make suggestions for improvement • Develop a common understanding of the Oregon quality review process • Develop a common understanding of the rating scale and descriptors for the four rubric dimensions and the rating categories and descriptors for overall ratings • Develop their abilities to use the criteria, rating scales and rating descriptors to accurately rate instructional materials

4. Oregon Quality Review: Principles & Agreements • CCSS:Before beginning a review, all members of a review team are familiar with the CCSS. • Inquiry:Review processes emphasize inquiry rather than advocacy and are organized in steps around a set of guiding questions. • Respect & Commitment:Each member of a review team is respected as a valued colleague and contributor who makes a commitment to the review process. • Criteria & Evidence:All observations, judgments, discussions and recommendations are criterion and evidence based. • Constructive:Lessons/units to be reviewed are seen as “works in progress.” Reviewers are respectful of contributors’ work and make constructive observations and suggestions based on evidence from the work. • Individual to Collective:Each member of a review team independently records his/her observations prior to discussion. Discussions focus on understanding all reviewers’ interpretations of the criteria and the evidence they have found. • Understanding & Agreement:The goal of the process is to compare and eventually calibrate judgments to move toward agreement about quality with respect to the CCSS.

5. Using the Quality Review Rubric

6. The Three Shifts in Mathematics • Focus: Strongly where the Standards focus • Coherence: Think across grades and link to major topics within grades • Rigor: Require conceptual understanding, fluency, and application

7. Activity 1:Finding Focus & Coherence

8. Looking for Focus • Taking a closer look at the major work of each grade • Two levels of focus: • What’s in/What’s out • The shape of the content that is in

9. Activity 1

10. Finding Focus in the CCSS

11. HS content

12. Activity 2a:Finding Rigor

13. Rigor: In Major Topics, Pursue Conceptual Understanding, Procedural Skill and Fluency, and Application • The CCSSM require a balance of: • Conceptual understanding • Procedural skill and fluency • Application of skills in problem solving situations • Pursuit of all three requires equal intensity in time, activities, and resources.

14. The three legged stool

15. How do the Standards signal Rigor? • Conceptual Understanding: 3.NF.1 Understand a fraction 1/b as the quantity formed by 1 part when a whole is partitioned into b equal parts; understand a fraction a/b as the quantity formed by a parts of size 1/b. • Procedural Skill and Fluency: 5.NBT.5 Fluently multiply multi-digit whole numbers using the standard algorithm. • Application: 7.NS.3 Solve real-world and mathematical problems involving the four operations with rational numbers.

16. Application • Real-world problems (single- and multi-step) • Non-routine problems • Varied problem types (see Tables in CC.OA progressions) • Enhance major work of the grade • Constructing models (6-12)

17. Conceptual Understanding • Problems can (and should sometimes) be brief • Explaining reasoning is one way to address conceptual understanding • Problems and exercises should be grade-level appropriate • Connections between representations are emphasized

18. Procedural Skill and Fluency • Purely procedural problems • Opportunistic strategies; writers are thoughtful about numbers used • Repeated practice • Procedures are built from conceptual understanding

19. Activity 2b: Focus & Rigor

20. Activity 2b:Looking for rigor in texts

21. Implement tasks that promote reasoning and problem solving • Dan Meyer: “Math Class Needs a Makeover”

22. Need to develop patient problem solvers • What are characteristics of a task that places: • A low-level cognitive demand on students? • A high-level cognitive demand on students? • What does it mean for students to be “patient” or “impatient” problem solvers? • How can task selection and implementationcondition students to be one of these types of problem solvers?

23. Layers of a math problem

25. Activity 3Practice using OR-IMET

26. Using the Quality Review Rubric

27. OR-IMET

28. Using the Quality Review Rubrichttp://tinyurl.com/odemath-omet For each dimension: • Make observations and suggestions related to criteria and evidence. • Determine a rating for each dimension based on checked criteria and observations. • Additional comments to improve the rating of the material in this section

29. Grouping of Math Practices Reasoning and Explaining 2. Reason abstractly and quantitatively 3. Construct viable arguments and critique the reasoning of others Modeling and Using Tools 4. Model with mathematics 5. Use appropriate tools strategically Seeing Structure and Generalizing 7. Look for and make use of structure 8. Look for and express regularity in repeated reasoning Overarching Habits of Mind of a Productive Mathematical Thinker 1. Make sense of problems and persevere in solving them 6. Attend to precision Adapted from (McCallum, 2011)

30. Look for evidence

32. Giving Feedback Writing effective feedback is vital to the Quality Review Process. Below are the four qualities of effective feedback. • Criteria-based: Written comments are based on the criteria used for review in each dimension. No extraneous or personal comments are included. • Evidence Cited: Written comments suggest that the reviewer looked for evidence in the lesson or unit that address each criterion of a given dimension. Examples are provided that cite where and how the criteria are met or not met. • Improvement Suggested: When improvements are identified to meet criteria or strengthen the lesson or unit, specific information is provided about how and where such improvement should be added to the material. • Clarity Provided: Written comment are constructed in a manner keeping with basic grammar, spelling, sentence structure and conventions.

33. Feedback Example #1: Mathematics This unit clearly targets three CCSS, which are noted in the overview. The overview also indicates which Standards for Mathematical Practice are central to the lesson. The activities throughout the unit present a balance of mathematical procedures and deeper conceptual understanding of the standards. The activities reinforce the standards and are well-connected to the content. I think the activities might be challenging with a large class with classroom management issues. Is this feedback criteria-based? Was evidence cited? Was there an improvement suggested? Is clarity provided?

34. Initial findingsPilot review SOESD & Hillsboro

35. Programs Reviewed – Summer 2014 Southern Oregon ESD Middle School Core Focus Connected Math 3 Agile Mind Go Math Engage NY (6-8) • Elementary School • Bridges (K-5) • Engage NY (K-5) • Math Expressions • My Math • Investigations (incomplete materials submitted)

36. Programs Reviewed – Summer 2014 Southern Oregon ESD Hillsboro Regional Review High School HMH HS math (unpublished) College Prep Math Pearson Math McGraw Hill Math CK-12 Engage NY (attempted - incomplete) • High School • HMH HS math (unpublished) • Big Ideas • College Prep Math • Core Plus • Pearson Math • Engage NY (attempted - incomplete)

37. Preliminary Results:Elementary (SUM 14)

38. Preliminary Results:Elementary (SOESD: SUM 14)

39. Preliminary Results:Middle School (SUM 14)

40. Preliminary Results:Middle School (SOESD: SUM 14)

41. Preliminary Results:High School (SOESD & HSD: SUM 14)

42. Preliminary Results:High School (SOESD & HSD: SUM 14)

43. Lessons Learned & Moving Forward

44. Lessons Learned from Sum 14 • Training & Calibration is difficult and non-trivial • Significant Refinement from SOSED to HSD • Importance of providing practice with real programs • Understanding quality criteria valuable regardless if doing a formal review • Spill over effect of understanding concepts like focus and rigor in a new context • Need to understand quality as materials are organized or created • Strong interest in this work • Need for both purchase and creation of materials • Need to continue in 2015

45. Looking ahead to 2014-15 • Establishment of review cohort • Provide training to 3-4 leaders from 6-7 regions in the state (~24 statewide) • Can review materials/facilitate reviews Sum 2015 • PLT conferences • First day general sessions/Second day breakouts (math) • Fall training on Finding, evaluating, & modifying resources • Multi-State OER collaborative • ~10 states have agreed to support the development of CCSS OER courses in ELA and Math (including Oregon) • Coordinated by CCSSO and Creative Commons • RFP Fall 2014, courses as early as Sum/Fall 2015

46. Questions? • Mark Freed Mathematics Education Specialist Oregon Department of Education mark.freed@state.or.us 503-947-5610