Presentation Transcript

1. 2014 MCAS-Alt:How to Submit a “Complete” Portfolio

   Karen Orlando MCAS-Alt Teacher Consultant
2. Topics Six main reasons portfolios were scored Incomplete Other reasons for an Incomplete score Checking for completeness Common questions Massachusetts Department of Elementary and Secondary Education
3. Top Six Reasons for Incomplete Portfolios Massachusetts Department of Elementary and Secondary Education
4. Top Six Reasons For Incomplete Portfolios Brief description(s) did not match the measurable outcome. (2,158 strands) Data chart did not document student performance of a single targeted skill/outcome. (1,958 strands) Data chart did not document student performance on at least eight different dates. (1,131 strands) No evidence of self-evaluation in the strand. (889 strands) At least two pieces of primary evidence were not submitted that related to outcome listed on the data chart. (714 strands) Brief descriptions of each activity were missing or did not provide enough information (666 strands) Massachusetts Department of Elementary and Secondary Education
5. Top Six Reasons For Incomplete Portfolios Brief description does not match the measurable outcome Massachusetts Department of Elementary and Secondary Education
6. Pages 32 and 35, Educator’s Manual for MCAS-Alt Brief Descriptions 2014 Educator’s Manual: Page 32: “…show the student’s performance of the measurable outcome (one specific skill) that is aligned with the strand required for assessment for a student in that grade.” Page 35 “…all learning activities match the measurable outcome being assessed.” Massachusetts Department of Elementary and Secondary Education
7. Top Six Reasons For Incomplete Portfolios Data chart did not document student performance of a single targeted skill/outcome. Massachusetts Department of Elementary and Secondary Education
8. Pages 12, 32, and 36, Educator’s Manual for MCAS-Alt Single Targeted Skill/Outcome 2014 Educator’s Manual: Page 12 “…measuring the student’s performance of one targeted skill” Page 32 “A minimum of one data chart and two pieces of additional primary evidence (called the “core set of evidence”) are required in each portfolio strand that together document one targeted skill (called the “measurable outcome”).” Page 36 “…only one skill is assessed in all of the evidence submitted in the strand.” Massachusetts Department of Elementary and Secondary Education
9. Using Entry Points to Create Measurable Outcomes that Address a Single Skill Entry points in the following subjects and grades may be used “as is” to create a measurable outcome. ELA: Grades preK-12 Math: Grades preK-8 All work and all data pointsmust address the same skill When an entry point in these grades includes more than one skill, teachers can either: modify the outcome to address only part of an entry point, if it is more appropriate for the student; OR Address all of the skills in the entry point on every date Massachusetts Department of Elementary and Secondary Education
10. How to Use an Entry Point that Includes Multiple Skills Example: Entry Point As Written: “Solve number sentences that represent one-step multiplication and division word problems” Option 1: Create a measurable outcome that addresses both skills. All work and data points must show “solving number sentences involving multiplication AND division” Option 2: Modify the outcome: “Solve number sentences that represent one-step multiplication word problems” All work and data points must show “solving number sentences involving multiplication.” Massachusetts Department of Elementary and Secondary Education
11. Using Entry Points to Address Single Skill The following entry points will often include multiple skills. Science grades: Grades preK-12 Mathematics: Grades 9-12 However, the teacher must select one skill within the entry point on which to base the measurable outcome. All work and data points must address that same skill. Massachusetts Department of Elementary and Secondary Education
12. Select One Skill Entry Point, as written (Science and Tech/Eng): “Identify and describe the purpose, function, and steps in the process of each of the following systems: digestive circulatory respiratory nervous muscular/skeletal sexual reproductive Teacher must select only ONE SKILLamong many options Multiple skills and systems may NOT be combined. Example: “Identify the steps in the process of the digestive system” Massachusetts Department of Elementary and Secondary Education
13. Top Six Reasons For Incomplete Portfolios Data chart did not document student performance on at least eight different dates. Massachusetts Department of Elementary and Secondary Education
14. Pages 36, Educator’s Manual for MCAS-Alt Possible Reasons for this Comment Either… 8 dates were included, but the same skill was not assessed on a total of 8 dates. Outcome was overly broad, and brief descriptions showed more than one skill. Dates were on weekends or holidays. 2014 Educator’s Manual: Page 36: “…the same skill was measured on at least eight dates.” Massachusetts Department of Elementary and Secondary Education
15. Top Six Reasons For Incomplete Portfolios No evidence of self-evaluation was found in this strand. Massachusetts Department of Elementary and Secondary Education
16. Pages 34, Educator’s Manual for MCAS-Alt Self-Evaluation Either… Missing altogether Examples show no evidence of student choice Stickers Stamp “Student smiled” “Student traded in for music” 2014 Educator’s Manual: Page 34: “Self-evaluation activities document the student’s choices, decisions, and involvement before, during, and after instruction, including evidence that the student performed” Massachusetts Department of Elementary and Secondary Education
17. Examples of Self-Evaluation include: planning and goal setting choosing an activity or next steps in an activity selecting a problem-solving strategy monitoring own progress or use of a strategy (e.g., by checking off steps as each is completed) deciding when to continue or end participation in an activity identifying and correcting his/her own errors graphing own performance or progress on a chart determining own score using a rubric selecting work for his/her own portfolio reflecting on his or her performance; for example, teacher asks student to respond to: What did I do well? What am I good at? Was this too easy? What did we do during this activity? What did I learn? How could I do better? Where do I need help? What should I work on next? What would I like to learn? Massachusetts Department of Elementary and Secondary Education
18. Top Six Reasons For Incomplete Portfolios At least two pieces of primary evidence were not submitted that assessed the outcome listed on the data chart. Massachusetts Department of Elementary and Secondary Education
19. Massachusetts Department of Elementary and Secondary Education
20. Pages 12 and 32, Educator’s Manual for MCAS-Alt Two pieces of primary evidence 2014 Educator’s Manual: Page 12 “…two additional pieces of primary evidence showing the student’s performance of the same targeted skill identified on the data chart” Page 32 “A minimum of one data chart and two pieces of additional primary evidence (called the “core set of evidence”)” Massachusetts Department of Elementary and Secondary Education
21. Top Six Reasons For Incomplete Portfolios Brief descriptions of each activity were missing or did not provide enough information. Massachusetts Department of Elementary and Secondary Education
22. The brief descriptions below each activity do not provide enough information to know what the student was asked to do, or how he/she did it. Massachusetts Department of Elementary and Secondary Education
23. Pages 32, Educator’s Manual for MCAS-Alt Brief Descriptions 2014 Educator’s Manual: Page 32 “…brief description beneath each data point…clearly describes how the student addressed the skill. (Note: The brief description must include the context of the activity; i.e., what was the student asked to do and how did they do it?). “One- or two-word descriptions will be insufficient to document the relationship between the activity and the measurable outcome and may exclude one or more data points from being scored.” Massachusetts Department of Elementary and Secondary Education
24. Good Brief Descriptions Measurable Outcome: Walter will identify simple machines with 80% accuracy and 100% independence. Describes the skill, activity, and context States how the student addressed the measurable outcome Massachusetts Department of Elementary and Secondary Education
25. Other Reasons for Incomplete Both Accuracy and Independence began above 80-100% Data were unclear or could not be interpreted by the scorer. No data chart was submitted. Student's name and/or date (month, day, year) was missing on one or more pieces of primary evidence, or was incorrect on Work Sample Description label. Percent accuracy and/or independence were not included on primary evidence and could not be determined by scorer. Evidence in ELA and Math was not produced during the current school year (i.e., since July 1, 2012). Massachusetts Department of Elementary and Secondary Education
26. Level of Complexity Score of 1 Massachusetts Department of Elementary and Secondary Education
27. Page 48, Educator’s Manual for MCAS-Alt What Does Level of Complexity = 1 Mean? Strand must assess standards required for a student in that grade. Skill must be taught in the context of an academic activity (especially if addressing Access Skills). Measurable outcome must reflect a standard or entry point in the Resource Guide. ELA Reading is comprehension of words, phrases, sentences, passages based on a text. Massachusetts Department of Elementary and Secondary Education
28. Skill Not Taught in the Context of an Academic Activity Massachusetts Department of Elementary and Secondary Education
29. Earth and Space Science is not assessed in High School Massachusetts Department of Elementary and Secondary Education
30. Page 5, ELA Resource Guide NEW for 2014 ELA Reading: Cite the Text! 2013 ELA Resource Guide: Page 5: “Text comprehension includes the understanding of words, phrases, and sentences in the context ofa text, rather than in isolation. Evidence in the MCAS-Alt portfolio for the Reading strand must include either a reference to, or a copy of, the actual text (sentence, passage, narrative, etc.) from which any words or phrases were selected. “ Text refers to information, ideas, stories, poetry, or portrayals of real or imagined events read by or to the student or conveyed in writing, pictures, Braille, or the student’s primary mode of communication. Text may be supported by symbols, graphics (e.g., illustrations, drawings), technology (e.g., voice output in an AAC device, computer, iPad), or three-dimensional objects.” Massachusetts Department of Elementary and Secondary Education
31. Checking for Completeness Massachusetts Department of Elementary and Secondary Education
32. Pages 35-36, Educator’s Manual for MCAS-Alt 7 Key Questions to Check for Completeness Is there a data chart and at least two pieces of primary evidence? Do the data chart and two pieces of primary evidence document student performance on one skill? Is the student’s name, date, % accuracy, and % independence listed on the data chart and on two pieces of primary evidence (or work description)? Massachusetts Department of Elementary and Secondary Education
33. 7 Key Questions to Check for Completeness (continued) Are the following listed on each data point and on two pieces of primary evidence: Student’s name, date, % accuracy, and % independence Do the brief descriptions tell what the student was asked to do on 8 valid dates? Does the first data point on the data chart begin below 80% for accuracy and/or independence? If photographs or videos are included, is the image clear and the final product or performance clearly labeled. Massachusetts Department of Elementary and Secondary Education
34. Other Common Questions Massachusetts Department of Elementary and Secondary Education
35. How Do I Determine Accuracy and Independence? First, determine what the outcome is. What are you asking the student to do? Then, determine the activity the student will perform to address that outcome. Next, divide that activity into “items” or steps. Determine the overall accuracy (correctness) and independence (no prompts needed) for the entire activity: % Independence = Number of “items” performed independently Number of possible responses Massachusetts Department of Elementary and Secondary Education
36. Page 38, Educator’s Manual for MCAS-Alt Examples of Items or StepsOutcome:Alex will identify the stages of the life cycle of a butterfly using pictures with 80% accuracy and 80% independence.Activity:The student will place pictures in order showing the stages of the life cycle of the butterfly. Massachusetts Department of Elementary and Secondary Education
37. Pages 37-39, 50 Educator’s Manual for MCAS-Alt Independence: Prompts vs. Accommodations The following are cues and prompts since they guide the student toward the answer: Verbalassistance (coach or tell: ”look on page…”) Gesture (point or indicate where to find answer) Visual cue (provide a sample problem or model) Full or partial physical assistance (including hand-over-hand assistance) Any Cue or Prompt =a “non-independent response” Massachusetts Department of Elementary and Secondary Education
38. How many questions? 5 How many correct responses? 2 P What is the accuracy score? 2/5 or 40% I I How many independent responses? 4 I What is the independence score? 4/5 or 80% I
39. Page 53, Educator’s Manual for MCAS-Alt What is Generalized Performance? 2014 Educator’s Manual: Page 53: “Generalized performance reflects the number of instructional approaches and activity formats through which the student acquires and demonstrates knowledge and skills, including any of the following: media and materials activity formats presentation formats methods of response” NEW for 2014: The GP score will NOT be increased based solely on homework. Massachusetts Department of Elementary and Secondary Education
40. What is a Teacher-Scribed Work Sample and When Should it Be Used? Student’s performance on a series of tasks recorded by the teacher on one date. One method of documenting performance of a student who does not produce written work Indicates what the activity was, what response options were offered to student, and how he/she responded on each item (accurately, independently?) Massachusetts Department of Elementary and Secondary Education
41. Good Example of Teacher-Scribed Work Sample What was the activity? What materials were used? What was the correct response? What was the student’s actual response? Massachusetts Department of Elementary and Secondary Education
42. Teacher-Scribed Work Sample: Not Enough Information What was the activity? What was the correct response? What was the student’s actual response? What materials were used? Massachusetts Department of Elementary and Secondary Education
43. Reminders Double-check evidence you are submitting Have another teacher or supervisor double-check your evidence Check for clerical errors such as incorrect dates or work of another student Attend portfolios-in-progress review session (January and March) to have training specialist look over your portfolios and share ideas Make sure you are using current forms, manuals, and Resource Guides Massachusetts Department of Elementary and Secondary Education