MCAS Alternate Assessment (MCAS-Alt) Collecting Instructional Data

1. MCAS Alternate Assessment(MCAS-Alt)Collecting Instructional Data October 2012 Karen Orlando MCAS-Alt Teacher Consultant

2. Why Are Data Charts Required for MCAS–Alt? Data charts provide: • Objective information on academic performance over a defined period of time • Graphic display of progress to share with others, including parents • Information on which to base the MCAS-Alt score • Accuracy • Independence • Generalized performance (i.e., brief descriptions of each activity) • instructional approaches, presentation and response methods 2013 MCAS-Alt: Data Collection

3. 2013 Educator’s Manual Pp.33 – 34, 43 – 45 Data Charts for MCAS-Alt 2013 MCAS-Alt: Data Collection • Each strand must include at least one data chart, either a: • bar graph • line graph • field data chart • Each data chart must include: • A measurable outcome (i.e., ONE targeted skill) based on the required strand or learning standard • At least 8 different dates on which the activity was taught, or skill was assessed • Accuracy and independence on each date • Brief description beneath each activity

4. Data Charts: How do I get started? 2013 MCAS-Alt: Data Collection • Blank print versions are available in the 2013 Educator’s Manual (See pp. 86-88) • Digital data charts are available online at:www.doe.mass.edu/mcas/alt/resources.html Other important Information: • Data collection for 2012-2013 may begin July 1, 2012. • Data for Science and Technology/Engineering (STE) can be collected during either 1 or 2 consecutive school years (current and previous school years).

5. See Educator’s Manual, pp. 37– 42 8-Step Data Collection Process 2013 MCAS-Alt: Data Collection • Determine the standards-based skill to be measured. • Develop an observable, measurable, and individualized outcome. • Individualize the instructional approach. • Select the type of data to be collected. • Set up a system for recording data on the student’s performance. 6.Collect data on a regular basis. 7.Monitor and analyze data regularly. 8. Make systematic instructional decisions based on data analysis.

6. Step 1: Determine the Skill to Be Measured Entry Point (below grade level) Standard As Written (at grade level) Access Skill (during academic instruction) High - Medium - Low complexity 2013 MCAS-Alt: Data Collection • Refer to Resource Guide in the content area and strand to be assessed. • Select a learning standard at student’s enrolled grade. • If not working at grade level, determine “entry point” at a challenging but attainable level of complexity.

7. How to Use the Resource Guideto Determine Skills to be Measured Ask whether the student can: • Address learning standards at grade level? • If not, address higher-complexity “entry points?” • If not, address lower-complexity “entry points”in the same topic, if possible? • If not, address other learning standards in the required strand (or ELA General Standard). If no to all of the above: • Address access skills during standards-based instruction (e.g., making choices, interacting) 2013 MCAS-Alt: Data Collection

8. How to Identify a Single Skill to Measure 2013 MCAS-Alt: Data Collection • Identify a strand required for MCAS-Alt Example: Life Science (Biology) • Then, select one standard you feel is appropriate to teach your student. Standard 4, grade 3-5(Resource Guide, page 266): Describe the major stages that characterize the life cycle of the frog and butterfly as they go through metamorphosis. • If necessary, review entry points, beginning with more complex. Describe the life cycle of a plant or animal (page 264) • Isolate a single skill to be assessed. Describe the life cycle of a butterfly.

9. How to Determine Level of Complexity 2013 MCAS-Alt: Data Collection • Pretest the student on the skill and collect baseline data. • If skill doesnotchallenge the student, increase level of complexity of the task. • If skill is too challenging for the student, adjust level of complexity. • When appropriate level of complexity is established (challenging and attainable), begin collecting data for the portfolio. • When student masters the skill, raise the level of complexity or address a different skill. • In either case, BEGIN A NEW CHART.

10. Baseline Data Requirement 2013 MCAS-Alt: Data Collection • Data charts must show that a new skill was taught, not a skill the student already can do. • Therefore, independence or accuracy (or both) MUST BEGIN below 80%. • Strand will be scored “M” (missing or insufficient) if a data chart begins at 80-100% in BOTH Accuracy ANDIndependence.

11. Example of a Scorable Data Chart 2013 MCAS-Alt: Data Collection

12. Example of an Unscorable Data Chart 2013 MCAS-Alt: Data Collection

13. How do you determine what skills to teach your students now? 2013 MCAS-Alt: Data Collection • Informal Assessments: • Observation • Parent and support staff input • Trial and Error • Formal Assessments: • Collecting data • Checklists

14. Important to Remember 2013 MCAS-Alt: Data Collection • Select only ONE SKILL to assess. • Learning standards and entry points often include multiple skills. • When scrolling back to entry points in earlier grades, select a related entry point, if possible. • Only a few, very important academic outcomes/skills are documented in the portfolio.

15. Determine Skill to Be Measured 2013 MCAS-Alt: Data Collection Learning Standard 4 Life Science: Biology Describe the major stages that characterize the life cycle of the frog and butterfly as they go through metamorphosis. Entry Point: describe the life cycle of a plant or animal (page 264) Skill: Describe the life cycle of a butterfly.

16. Step 2: Develop a Measurable and Observable Outcome for Student • How will we know when the student • has mastered the skill? • First, determine the skill: • Describe the life cycle of a butterfly. • Next, add: • Alex will identify the stages of the life cycle of a butterflyusing pictures with 80% accuracy and80% independence. • conditions • criteria 2013 MCAS-Alt: Data Collection

17. Observable vs. Non-observable Skills 2013 MCAS-Alt: Data Collection • Jeff will identify whole numbers from one to ten. • Jeff will know all the whole numbers between one and ten. • After listening to a story, Lex will answer comprehension questions. • Lex will understand stories read to him. • Taylor will sort shapes by color. • Taylor will explore different shapes.

18. Making the Skill Measurable 2013 MCAS-Alt: Data Collection Skill: Shana will identify synonyms. Add specificity and a measure of frequency to create a measurable outcome: Shana will identify the synonyms of words highlighted in a text with 75% accuracy and 100% independence. OR Shana identify the synonyms of words highlighted in a text with 75% accuracy and 100% during 3 consecutive days.

19. Step 3: Individualize the Instructional Approach 2013 MCAS-Alt: Data Collection • What type(s) of instruction are best-suited to the skill being measured? • Use Systematic Instruction Techniques • Time Delay • System of least prompts • Consider: • Adaptations/modifications • Student preferences (presentation and response) • Activity format and materials • With whom and where will instruction occur? • How frequently will data be collected?

20. Educator’s Manual p. 39 Independence: Prompts vs. Accommodations 2013 MCAS-Alt: Data Collection All of the following are cues and prompts when they guide the student toward the answer: • Verbalassistance (coach or tell: ”look on page…”) • Gesture (point or indicate where to find answer) • Visual cue (provide a sample problem or model) • Full or partial physical assistance (including hand-over-hand assistance) Remember: Any cue or prompt = Non-independent response

21. Calculate Independence Correctly Measurable Outcome: Student will order each stage of the butterfly life cycle. If student was prompted on one stage, Independence should have been 75%, not 95% (3 out of 4 were done independently) 2013 MCAS-Alt: Data Collection

22. Accommodations 2013 MCAS-Alt: Data Collection Accommodations do not affect the percent of Independence. Examples of Accommodations: • Redirection to task • Positive Reinforcement Systems • Scribing • Reading text to student for comprehension

23. Step 4: Determine Accuracy and Independence 2013 MCAS-Alt: Data Collection • First, determine the outcome. - What are you asking the student to do? • Then, determine the activity during which the student will work on that outcome. • Next, break that activity into steps or tasks. • Determine the accuracy (correctness) and independence (performed without one or more prompts) for each step or task in the activity.

24. Step 5: Set Up a System to Record Data on the Student’s Performance 2013 MCAS-Alt: Data Collection Select/adapt or design a data sheet that: • Matches the instructional approach • Is easy to use by teacher(s) and other staff • Is clear enough to be shared with others (e.g., parents) • Includes all required information

25. Example: Set Up a System for Recording Data on the Student’s Performance 2013 MCAS-Alt: Data Collection • Targeted skill:“Identify numbers 1-10” • Measurable outcome: Student will identify the numbers 1-10 with 75% accuracy and 100% independence during 3 consecutive math sessions. • Type of Data: Response-by-Response • Student verbally identifies number presented • Several attempts by student during a single session • System for Recording Data: Field Data Chart • Teacher will be able to document each response most effectively on a field data chart.

26. Guidelines: When to Use Each Type of Data Chart 2013 MCAS-Alt: Data Collection Field Data Chart: • Several related tasks in one or more settings on a single date • Task does not result in a paper/pencil product • Data collected at the moment student is participating in the activity Remember: Measure one skill per chart. Use a new chart each time the skill changes.

27. Sample Field Data Chart 2013 MCAS-Alt: Data Collection 2013 MCAS-Alt: Data Collection

28. Guidelines: When to Use Each Type of Data Chart/Graph 2013 MCAS-Alt: Data Collection Line Graph: • Shows same information as a bar graph • Summarizes student performance over time on a single skill • Shows progress more clearly (trend), but may be more difficult to see performance on single dates • May be more difficult than bar graph for parents or students to read (e.g. crossing lines)

29. Sample Line Graph 29 2013 MCAS-Alt: Data Collection

30. Guidelines: When to Use Each Type of Data Chart 2013 MCAS-Alt: Data Collection Bar Graph: • Summarizes student performance over time • Shows progress “at-a-glance,” though sometimes difficult to tell from solid bars if student is making progress. Note: You may want to continue collecting data beyond 10 dates. If so, start a new graph.

31. Sample Bar Graph 2013 MCAS-Alt: Data Collection

32. Step 6: Collect Data on a Regular Basis 2013 MCAS-Alt: Data Collection • Decide whether daily, weekly, or even monthly data collection is best suited to the student and skill being assessed. • Once a schedule is established, be consistent with collecting data • Avoid teaching same skill the same way, and at same time, each day. • Important to see if skill can be generalized across different activities and settings

33. Teacher-Scribed Work Sample 2013 MCAS-Alt: Data Collection • When a paper-and-pencil response is not possible, one option may be to scribe all of the student’s responses to the activity on a single date. • Counts as primary evidence (work sample), if it: • Provides information on the activity; • i.e., format and materials used • Describes what student was asked to do • Shows the student’s responses (accuracy) • Shows level of prompts (independence). • Okay to use a key (+/— and I/P)

34. Example: Teacher-scribed work sample: A series of tasks recorded by the teacher on one date. 2013 MCAS-Alt: Data Collection

35. Step 7: Monitor and Analyze Data on a Regular Basis • Review data regularly. • Is the student making progress? • If not, are instructional changes needed? • Share relevant data with all members of the student’s IEP Team and support staff. 2013 MCAS-Alt: Data Collection

36. Step 8: Review Data and Make Instructional Decisions • Use data to evaluate instruction and make necessary changes • Determine whether to increase or decrease the complexity of the skill? • Should a different approach be used? • Good instructional decisions based on data will lead to student success 2013 MCAS-Alt: Data Collection

37. Data Chart Reminders Only ONE SKILL on a data chart Must have at least 8 separate dates listed on data chart Each data point must include a brief description that is clear, concise, and specific How did the student address the measurable outcome? “The student (did what?) (How?)” Baseline (starting point) on data chart must be below 80% accurate or 80% independent. 2013 MCAS-Alt: Data Collection

38. Data Chart+ at least 2 pieces of additional primary evidence, all showing the same skill =A Complete Strand 2013 MCAS-Alt: Data Collection