Waiting Room

1. Waiting Room Today’s webinar will begin shortly. REMINDERS: • Dial 800-503-2899 and enter the passcode 6496612# to hear the audio portion of the presentation • Download today’s materials from the sign-in page: • Webinar Series Part 7 PowerPoint slides • Using Current Assessments in DDMs Massachusetts Department of Elementary & Secondary Education

2. Ramping Up for Next Year:Strategies for Using Current Assessments in DDMs Webinar Series Part 7

4. Audience & Purpose • Target audience • District teams that will be engaged in the work of identifying, selecting, and piloting District-Determined Measures. • After today participants will understand: • Strategies for using current assessments as a starting place for District Determined Measures

5. Educator Evaluation Data SY 2012-2013

6. Data Release – 11/21/13 • 2012-13 educator evaluation performance rating data released for 213 RTTT districts • 37,940 educators • Majority of Educators Received Solid Ratings • 85.2% Proficient • Early evidence of differentiation is encouraging • 7.4% Exemplary; 6.8% Needs Improvement; and 0.7% Unsatisfactory

7. Ramping Up for DDM Implementation Identifying DDMs beyond the minimum pilot areas

8. Key Messages for Stakeholders

9. Ramping Up • Goal by June 2014: To identify at least two measures of student growth to match with each educator for use in the 2014-15 school year.

10. Ramping Up • Strategies: • Collaborate with other districts (e.g. network at the upcoming TA sessions, work with area collaboratives) • Engage educators in the process of identifying DDMs • Use assessments currently in use in the district as a starting point for DDMs

11. Why use Current Assessments? • Feasibility • Potentially cost effective • Familiarity and experience with the assessments • They are Aligned to Content • Assess what is most important for students to learn • Assess what the educators intend to teach • They are Informative • Teachers already use the assessments to measure student learning • Final Step • Provide valuable information to schools and districts about an educator’s impact

12. Considerations in DDMs • Performance vs. Growth • Most current assessments are measures of performance and not growth • Growth takes into account the different levels of student achievement • Measures of growth should provide all students an equal opportunity to demonstrate growth • Unit vs. Year • Most current assessments are designed to assess short units • DDMs that represent a full year provide more accurate representations of student growth and of an educator’s impact

13. Potential Solutions • Bring together multiple assessments of similar content to measure performance at different points in time • Administer one assessment (or similar versions) at multiple points during the year • Combine multiple assessments of growth into a single DDM (e.g. pre-post tests of multiple units)

14. The larger DDM context

15. New Guidance from ESE • Using Current Assessments in District-Determined Measures: Leveraging the Curriculum-Embedded Performance Assessments From the Model Curriculum Units • Available at http://www.doe.mass.edu/edeval/ddm/UsingAssessments.pdf

16. Using Current Assessments: Three Steps • Identify key content. • Ensure that change in performance represents student growth. • Select an approach for measuring growth.

17. Using Current Assessments: Three Steps • Identify key content. • Ensure that change in performance represents student growth. • Select an approach for measuring growth.

18. Step 1: Identify Key Content • Key Content may include content that is: • A meaningful sample of the learning • Challenging to teach and learn. • Under represented in assessment • District priorities.

19. Example: 3rd Grade ELA • Looking across the curriculum map to identify a meaningful sample of content

20. Example: 3rd Grade ELA

21. Example: 3rd Grade ELA • Standard 3 reads (W3.3) Write narratives to develop real or imagined experiences or events using effective technique, descriptive details, and clear event sequences.

22. Example: 9th Grade US History • Are there commonalities in content covered across multiple units aligned to the MA Frameworks.

23. Example: 9th Grade US History

24. Step 1: Identify Key Content • Key content may be taught repeatedly across the year • Key content may be taught once during the year

25. Step 1: Identify Key Content Question for Engaging Educators • What can a student do after they take your class that they couldn’t do before?

26. Using Current Assessments: Three Steps • Identify key content. • Ensure that change in performance represents student growth. • Select an approach for measuring growth.

27. Step 2: Does change represent growth? What is Growth? • Educators understand growth. • Growth is about improvement and learning. • There is a difference between growth and how we measure growth. • We can measure growth in both simple and sophisticated ways.

28. Step 2: Does change represent growth? • Are the assessments similar enough to support meaningful inferences about student growth during the year? • Do early assessments provide meaningful information about what students do not understand? • Do future assessments provide meaningful information about what students have learned? • Do students have the opportunity to demonstrate different levels of growth?

29. Example: 3rd Grade ELA • In the third grade example • Each unit has a CEPA covering the same content • Rubrics are different, hard to compare • Solution • Identify similar evidence • Modify rubrics by adding consistently worded rubric items that assess the same content

30. Looking Across Assessments

31. Example: 3rd Grade ELA • Conclusion: • The rubrics are similar enough to conclude that change in performance represents growth. • Advantages: • Do not need to change the tasks students will complete • Rubrics are on a consistent scale • Rubric do not need to be identical • Still include important rubric items that are present in only one CEPA (for example, use of technology in the Newspaper Unit)

32. Example: 9th Grade US History • Constitutional Rights: Presentation where students take on different roles • Economies of North & South: Write a journal, letter, … etc. • Industrial Revolutions: Persuasive Writing • Westward Expansion: Work in small groups making exhibit

33. Example: 9th Grade US History • Conclusion: • The performance tasks are too dissimilar to conclude that change in performance represents growth. • Next Steps: • Identify a way to assess the key content consistently. • For example, administer a single assessment of the key content at multiple times during the year.

34. Using Current Assessments: Three Steps • Identify key content. • Ensure that change in performance represents student growth. • Select an approach for measuring growth.

35. Step 3: Select an approach • Think about the scoring approach that will best capture student learning given the type of measure you have chosen. • For example: • Pre-Test/Post Test • Repeated Measures • Holistic Evaluation • Post-Test Only Learn more Webinar 5 http://www.doe.mass.edu/edeval/ddm/webinar.html Technical Guide B http://www.doe.mass.edu/edeval/ddm/TechnicalGuide-AppxB.pdf

36. Example: 3rd Grade ELA • Each student has five time points. • Classroom teacher rates all five time points, which provides information to parents and teachers about growth across the year. • First and last assignments are rated by two additional teachers. The difference is used to create a growth score. (The sum of the four rubric items. They decide to set standards after the pilot year.)

37. Example: 9th Grade US History • Each student completes four writing assignments. • The two US History teachers create a holistic rubric that looks at four aspects of the writing. • The two US History teachers flag disagreements between ratings and meet to resolve these differences.

38. More Information • Using Current Assessments in District Determined Measures: Leveraging the Curriculum-Embedded Performance Assessments From the Model Curriculum Units • Available at http://www.doe.mass.edu/edeval/ddm/UsingAssessments.pdf.

39. Closing

40. Attend Next Week’s TA Session • Date: December 12, 2013 • Time: 2:30pm – 5:30pm • Locations: Southeastern MA: Norwell High School, Little Theatre, 18 South Street, Norwell, MA 02061 • Northeastern MA: Groton-Dunstable Regional High School, 703 Chicopee Row, Groton, MA 01450 • Western MA: Frontier Regional School, Media Center, 113 North Main Street, South Deerfield, MA 01373 • Register: http://www.surveygizmo.com/s3/1462056/Registration-DDM-Technical-Assistance-Networking-Session-3-December-12th-2013

41. Sharing Locally-Developed DDMs • We need your help! Please submit: • Assessment directions, materials, items and prompts • Scoring resources (rubrics, answer keys) • Other information (steps for administration, lesson plans, examples of how results are used) • To submit your examples, complete the survey and upload your materials at https://app.smartsheet.com/b/form?EQBCT=52e3762ac7954f93aa40cc879bc38855.

42. Register for Webinar Series Part 8 • Part 8: Communicating Results • Date: January 23, 2014Time: 4-5pm EST (60 minutes)Register: https://air-event500.webex.com/air-event500/onstage/g.php?d=596783122&t=a

43. Questions • Contact • Craig Waterman at cwaterman@doe.mass.edu • Ron Noble at rnoble@doe.mass.edu • Tell us how we did: http://www.surveygizmo.com/s3/1469270/District-Determined-Measures-Assessment-Literacy-Webinar-7-Feedback