David W. Snyder, Illinois State University dsnyder@ilstu

1. Music Student Teachers’ Perceptions of the edTPA’s Effectiveness in Assessing their Teaching Abilities David W. Snyder, Illinois State University dsnyder@ilstu.edu

2. What is the edTPA? • a model for student teacher assessment that follows the portfolio model put forth by the National Board of Professional Teaching Standards (NBPTS) and developed from a California-based assessment system called the Performance Assessment for California Teachers (Darling-Hammond, Newton, and Wei, 2012). • The edTPA includes a review of a teacher candidate's teaching materials and short videos of the candidate teaching to document and demonstrate a candidate's ability to effectively teach his/her subject matter to all students (edtpa.aacte.org). • student teacher candidates are assessed in the areas of planning, instruction and assessment using reflective writing assignments and video critiques of their own teaching. These performance–based portfolios are then assessed by outside evaluators who are themselves professional educators trained by Pearson. (edTPA K–12 Performing Arts Assessment Handbook, 2013).

3. Where are we on implementing the edTPA? As of February 2014, there are 34 states who have either piloted it or are currently using it as an assessment Of those, six states (Hawaii, Washington, Minnesota, Wisconsin, Tennessee, and New York), currently require all their student teachers to take it Five other states (California, Illinois, Ohio, Massachusetts, and Georgia) will soon require or offer edTPA as an assessment option The remaining 23 states have at least one institution using the edTPA

4. Where are we on implementing the edTPA?

5. Where are we on implementing the edTPA? • Washington state required teacher candidates to take edTPA to complete teacher preparation programs and receive licenses for most subject areas starting this January. • In New York, starting in May 2014 teacher candidates applying for licenses will be required to pass edTPA. • Illinois will require the edTPA for teacher licensure starting the fall of 2015. • Georgia will require edTPA for teacher licensure beginning in 2015-16. • Wisconsin will require edTPA for teacher licensure starting in 2016-17. The assessment will be used as one measure for teacher preparation program approval beginning in 2015-16. • California is piloting the edTPA as a fourth option among already required teacher performance assessments. • Minnesota's Board of Teaching, which approves institutions and licensure programs to prepare teachers, now uses edTPA as one measure. • Hawaii will require a performance-based assessment such as edTPA for teaching candidates beginning in 2017. • Tennessee now allows edTPA to be used as a substitute for a standardized test of teaching practice. -Doug Orzolek, Univ. St. Thomas from USA Today , Oct 2013

6. How is the assessment scored? • There are 3 tasks to be completed during the student teaching semester: Planning, Instruction and Assessment • There are 5rubrics within each task with a possible high score of 5 for each one • The recommended range of scores for passing by SCALE (Stanford Center for Assessment, Learning and Equity) is 37-42.

7. How is the assessment scored? PLANNING Rubric1 Planning for Developing Student Knowledge Rubric 2 Planning to Support Varied Student Learning Needs Rubric 3 Using Knowledge of Students to Inform Teaching and Learning Rubric 4 Identifying and Supporting Language Demands Rubric 5 Planning Assessments to Monitor and Support Student Learning

8. How is the assessment scored? INSTRUCTION Rubric 6 Learning Environment/Rapport Rubric 7 Engaging Students in Learning Rubric 8 Deepening Student Learning Rubric 9 Subject Specific Pedagogy/Modeling Rubric 10 Analyzing Teaching Effectiveness

9. How is the assessment scored? ASSESSMENT Rubric 11 Analysis of Student Learning Rubric 12 Providing Feedback to Guide Further Learning Rubric 13 Student Use of Feedback Rubric 14 Analyzing Students’ Language Use Rubric 15 Using Assessment to Inform Instruction

10. What is required to pass? • SCALE (Stanford Center for Assessment, Learning and Equity) developed the edTPA and is recommending a range of scores from 37-42 for passing the edTPA. • Different states are handling it differently: • New York has set the score at 41 to pass • Illinois will more than likely choose 39 or 40 as a passing score • Washington is starting at 35 and raising it in increments each year Elisa Palmer, Illinois State University

11. What is required to pass? Passing rates from second large-scale field test in 2013 by SCALE

12. Background on this current study All student teachers from the fall semester of 2013 (N=15) at ISU participated All students were assessed by official Pearson scorers AND completed a self-evaluation using the same rubrics In addition, all subjects completed an exit survey soliciting opinions on the helpfulness of the edTPA in preparing them for teaching Results from this study will be compared to the second large scale field test that was done in 2013 of teacher education candidates in fine arts completed by SCALE (N= 116)

13. How did we do? Passing rates from current study Fall 2013 (N=15) Scores ranged from 38-52 Mean score 44.7 Cut score of 38 100% passed Cut score of 39 94% passed Cut score of 40 94% passed

14. How did we do? 2013 Fall student teachers Pearson results: K–12 Performing Arts (N=15) Lowest score in red, highest in green Task 1: Planning Rubric 1 2 3 4 5 Mean 3.20 3.36 3.00 2.93 3.06 Std 0.41 0.48 0.68 0.45 0.17 Task 2: Instruction Rubric 6 7 8 9 10 Mean 3.16 2.90 3.00 2.86 2.80 Std 0.36 0.28 0.68 0.54 0.41 Task 3: Assessment Rubric 11 12 13 14 15 Mean 2.90 3.20 2.36 2.70 3.23 Std 0.47 0.79 0.61 0.62 0.70

15. How did they think they would do? 2013 Fall student teachers self-assessment: K–12 Performing Arts (N=15) Lowest score in red, highest in green Task 1: Planning (agreed on lowest score) Rubric 1 2 3 4 5 Mean 3.73 3.46 3.53 3.33 3.40 Std 0.70 0.83 0.83 0.90 0.91 Task 2: Instruction (agreed on highest score) Rubric 6 7 8 9 10 Mean * 4.00 3.33 3.13 3.40 3.53 Std 0.37 0.48 0.51 0.82 0.83 Task 3: Assessment (did not agree) Rubric 11 12 13 14 15 Mean * 3.60 3.46 3.062.93 3.33 Std 0.47 0.79 0.61 0.62 0.70

16. How have others done? Performing Arts results from second large-scale field test in 2013 (SCALE) 2013 Field Test Content Areas: K–12 Performing Arts (N=116) Lowest score in red, highest in green Task 1: Planning (agreed with self assessments) Rubric 1 2 3 4 5 Mean 3.09 2.99 2.93 2.89 2.91 Std 0.48 0.67 0.64 0.62 0.64 Task 2: Instruction (agreed with Pearson assessment) Rubric 6 7 8 9 10 Mean 2.95 2.74 2.76 2.89 2.46 Std 0.69 0.60 0.67 0.52 0.64 Task 3: Assessment (agreed on lowest score in Pearson assessment) Rubric 11 12 13 14 15 Mean* 2.83 3.022.34 2.49 2.73 Std 0.64 0.69 0.66 0.60 0.84

17. How have others done? • Rubric 4 (Identifying and Supporting Language Demands) • Rubric 10 (Analyzing Teacher Effectiveness) • Rubric 13 (Student Use of Feedback) • Were the three lowest scoring rubrics in the Planning, Instruction and Assessment Tasks respectively in both the current study (N=15)and the national pilot study in 2013 (N=116)

18. How have others done? • Rubric1 (Planning for Developing Student Knowledge) • Rubric 6 (Learning environment/Rapport) • Were the highest scoring rubrics in the Planning and Instruction Tasks in both the current study (N=15)and the pilot study in 2013 (N=116) • The Assessment Task rendered rubric 15 as the high score for the students in this study (N=15) and rubric 12 as the high score for the 2013 pilot study (N=116)

19. How have others done? • Doug Orzolek’s students at University of St. Thomas in Minnesota have been using this test for several years now and have the same three rubrics as their lowest average scores. • Rubric 4 Identifying and Supporting Language Demands • Rubric 10 Analyzing Teacher Effectiveness • Rubric 13 Student Use of Feedback

20. So why the problems with these rubrics (4, 10, 13) ? The language demand portion of the assessment (rubric 4) is very unclear in the handbook and in scorer training… students are asked to pick one “language function” for all 3 lessons, identify supporting vocabulary, and identify an additional language demand, either syntax or discourse

21. So why the problems with these rubrics? Language function should be the verb used in the learning objective, e.g., identify, analyze, compare, perform, create, etc. The students being taught need to demonstrate their understanding of the vocabulary necessary for the lesson in some way Student teachers have difficulty engaging in discourse “class discussion”…(student to student, teacher to student) focused on the main learning objective using proper vocabulary Nobody knows what syntax is “conventions for organizing symbols”…this can include the governing principles of reading music, i.e., rests, notes, musical staff

22. So why the problems with these rubrics? Rubric 10 requires student teachers to analyze their videos and propose changes to their instruction related specifically to student learning NOT to delivery and classroom management. Rubric 13 requires student teachers to explain specifically how the feedback given to their students will be used to revise their current work Writing “more practice” will not result in a high score Many student teachers don’t see their students again after completing the video sessions or they move on to a new piece or unit so they simply don’t address this in their commentary

23. What helped students prepare for the edTPA? Which of the support measures provided by ISU was most helpful to you in completing the edTPA? Please rank in order from 1 (most helpful) to 5 (least helpful) or DNA (did not apply). __Student teacher seminars __Feedback from university supervisor __Using the rubrics during a previous method class X On-line edTPA handbook __Deadlines provided during the semester

24. What did they think about the edTPA? Which set of tasks in the edTPA process did you find most difficult to complete and follow? Please rank from 1 (least difficult) to 3 (most difficult). X Planning __Instruction __Assessment

25. What did they think? Which set of tasks in the edTPA helped you improve your teaching the most?Please rank from 1 (most helpful) to 3 (least helpful).a. X Planning b. __ Instruction c. __ Assessment

26. What did they think? Which of the following commentaries/tasks under “Planning” was most difficult for you to complete and evaluate? Please rank from 1 (least difficult) to 5 (most difficult). __Rubric 1 developing student knowledge __Rubric 2 supporting student learning needs __Rubric 3 knowledge of students informing teaching X Rubric 4 language demands __Rubric 5 planning assessment This matches the actual results from both their self-assessment and the Pearson assessment

27. What did they think? Which of the following commentaries/tasks under “Instruction” was most difficult for you to complete and evaluate? Please rank from 1 (least difficult) to 5 (most difficult). __Rubric 6 learning environment/rapport __Rubric 7 engaging students in learning X Rubric 8 deepening students learning __Rubric 9 subject-specific pedagogy/modeling __Rubric 10 analyzing teaching effectiveness This DOES NOT match the actual results from the Pearson assessment Rubric 8 requires student teachers to prompt their students to higher order thinking or application related to the learning objective (questioning.) Students in this study reported difficulty distinguishing the difference between rubric 7 and rubric 8.

28. What did they think? Which of the following commentaries/tasks under “Assessment” was most difficult for you to complete and evaluate? Please rank from 1 (least difficult) to 5 (most difficult). __Rubric 11 analysis of student learning __Rubric 12 providing feedback __Rubric 13 student use of feedback X Rubric 14 analyzing student’s language use __Rubric 15 using assessment to inform instruction This DOES NOT match the actual results from the Pearson assessment Rubric 14 is the other language related rubric and has the same problems that rubric 4 does (misunderstanding of terms). Many students simply fail to assess the identified “language function” in their student assessments.

29. What did non-music students think? According to Darling-Hammond and Hyler in 2013, “the vast majority of candidates and teacher educators feel this process (edTPA) strengthens their practice”.

30. What did these music students think? Do you believe that participating in the edTPA process strengthened your practice as a music teacher? 1 (strongly agree) 2 (agree) 8 3 (disagree)6 4 (strongly disagree)1 Mean: 2.53 SD: .64

31. Questions? David W. Snyder dsnyder@ilstu.edu This PowerPoint presentation can be found at: www.cfa.ilstu.edu/dsnyder/news