Music Teacher Evaluation in Michigan Dr. Phillip M. Hash, Calvin College pmh3@calvin

1. Music Teacher Evaluation in MichiganDr. Phillip M. Hash, Calvin Collegepmh3@calvin.edu February 14, 2013

2. Overview of PM Workshop • New Legislation • Current Trends • Evaluation strategies • Assessment Strategies • Your Experience

3. Legislative Review • All Teachers Evaluated Annually • Percentage of Evaluation to Relate to Student Growth • National, State, And Local Assessments • Evaluations vs. Seniority in Personnel Decisions • Michigan Council On Educator Effectiveness 2

4. MDE Will Provide • Measures For every educator, regardless of subject taught, based on 2009-10 and 2010-11 data: • Student growth levels in reading and math • Student proficiency levels in math, reading, writing, science, social studies • Foundational measure of student proficiency and improvement (same for each teacher in a school) Understanding Michigan's Educator Evaluations, MDE (December 2010) • How will this data be used for arts educators? • Currently up to school districts • Might be specified by the state after this year

5. Performance-Based Compensation • A district shall implement a compensation method for teachers and administrators that includes “job performance and job accomplishments as a significant factor” to determine “compensation and additional compensation.” MCL 380.1250(1) • Meaning for arts educators?

6. New Prohibited Bargaining Subjects • 1. Teacher Placement • 2. Reduction in Force/Recall • 3. Classroom Observation • 4. Performance Evaluation • 5. Teacher Discharge/Discipline • 6. Performance-Based Compensation • 7. Parent Notification

7. Pilot Programs • 2012-13 Pilot • 14 districts • 4 evaluation models • Standardized tests • Local measures for non-tested subjects • Recommendations by 2013-14 school year 4

8. Current Trends in MI Teacher Evaluation

9. Frameworks, Methods, Systems Used as part of Local Evaluation

10. % Student Growth Counted in Teacher Evaluation (2011-12) % of Growth in Local Evaluation Systems

11. Current Trends: Effectiveness Ratings for 2011-12

12. Teacher Ratings & Student Growth

13. Evaluation Strategies

14. Evaluation Strategies • Always have lesson plans connecting to standards • See MI GLCE • Incorporate as many standards as make sense for your class – but not just perform and read notation • Study the evaluation form • Plan lessons using evaluation rubric as a guide • Be prepared to provide evidence of instructional & professional practices • Student work, rubrics, lesson plans, parent call log, etc. • Use a variety of instructional practices. • Focus on student engagement. • Don’t try to put on a show for evaluator • [Is it time to reconsider the number of performances per year??]

15. Danielson Example

16. Student Engagement in Rehearsalhttp://cart.bravomusicinc.com/ • http://www.youtube.com/watch?v=VY3nJXChWrY (student led warm-ups - breathing) • http://www.youtube.com/watch?v=tgdksldrwkc (chorale) • http://www.youtube.com/watch?v=_eOjYt1-4-0 (student sectionals - feedback)

17. Developing Local Assessment Strategies

18. Creating an Assessment Plan • District Music Faculty (by area) • Est. curriculum based on MI Standards • What should students in each grade level know and be able to do? • How and when will objectives be assessed? • Perhaps not every grade every year • How will assessments show growth? (e.g., difference in % b/w pre- post test, defined NP, PP, P, HP?) • Take plan to administration for approval • Law says that “with the involvement of teachers” • Pilot, Review, Revise, Implement

19. MI Grade Level Content Expectations(June 2011) • What students should know and be able to do in grades K-8, & HS • Aligned w/ VPAA & 21st century skills • Standards, & benchmarks by grade level • Teachers evaluated on use of standards • [See handout]

20. Assessment Terms • Reliability = Consistency • Test/retest (regardless of yr., location, etc.) • Interrater (every judge the same) • Validity = the extent to which an assessment measures what they purport to measure • Authentic Assessment = Students demonstrate knowledge and skills in real-world context (e.g., performance) • Quantitative – data is numerical (anything that can be counted, percentages) • Qualitative – data is in words (descriptions, written critiques) • Formative vs. Summative – • Formal vs. Informal -

21. Assessment Terms - RTTT • Rigorous • assessments that measure grade-level standards • Two points in time • pre- & post-test • Proficiency from one year to the next • Ongoing assessments of musical skills (steady beat, pitch matching, singing, recorder, instrumental performance, sight-reading, etc.) • Comparable across classrooms • same for all teachers at a particular level or area • Assessments comparable in rigor to other subjects

22. Student Growth Measures

23. Rubistarhttp://rubistar.4teachers.org/ • Create rubrics using existing descriptors • Search other teachers’ rubrics for samples • Edit to fit your needs

24. Rubrics • Types include: • Holistic (overall performance) • Analytic (specific dimensions of performance) • Additive • Descriptors must be valid (meaningful) • Scores • Must be reliable (consistent) • Should relate to actual levels of students learning • Can be used by students for self-assessment and to assess the performance of other students • Give to students b/f assessment 14

25. What does a rubric look like? • Features: • Scale includes rating points (at least 4). See handout for sample headings • Highest point represents exemplary performance • Criterion—based categories • Descriptors are provided for each level of student performance • Pre- and/or Post-test. Teacher, peer, & self assessment Adapted from: K. Dirth, Instituting Portfolio Assessment in Performing Ensembles, NYSSMA Winter Conference, Dec. 2, 1997. 13

26. Holistic Rubric

27. Holistic Rubric

28. Piano Rubric - Analytic Quiz #1 Scales Two octaves, hands together, ascending and descending Keys ____________

29. Sample Rating Scale 12

30. Showing Growth w/ Rubrics (or any other pre- post-test) • Pre- & post-test • average class posttest % - average class pretest % = % growth

31. Est. Personal Reliability • Record 10 students • Grade w/ rubric • Grade again in 2 weeks • Measure the difference in score for each recording • Calculate average difference • Lower = better

32. Rate these 6 recorder performances on a scale of 1-12Rate the same examples using rubric in handout Trial 1 1 _____ 2 _____ 3 _____ 4 _____ 5 _____ 6 _____

33. Recorder Trial 2 • Use rubric on loose sheet • Add up score • Match score from Trial 1 to Scores from Trial 2 • Is there a difference? • In which scores are you most confident?

34. Elementary General Music – Grade 3 Pre- & Post Test Sample • [See handout] • Paper/pencil, but relies on musical response • Prompts can be different for pre-test • Pre-test can be an abbreviated version • Require 2-3 class periods to complete • Music supervisor could issue musical examples & prompts before the test (avoid teaching to the test)

35. Creating Similar Elementary General Music Assessment • For grades 3-5, determine what GLCEs can be measured through paper/pencil response • Create question(s) for each benchmark – deliberately connect question to GLCEs (validity, rigor, comparable a/c classrooms) • Decide # of questions needed to determine competency • Create questions that fit different prompts

36. Performing Ensembles • Semester Exam [see handout] • Jason Lowe – Bay City HS Bands • Mandy Smith – Rockford HS Choirs

37. Watkins – Farnum Performance Scale • Sight reading – band • Published by Hal Leonard • Reliable & valid assessment • Forms A & B • Easy to score as per directions in handout • 14 exercises worth X pts. • Score until student earns 0 on 2 consecutive exercises

38. Royal Conservatory Music Development Program (see handout) • Recorder, strings, woodwinds, brass, percussion, voice • Graded preparatory, 1-10 • RC Grade 8 considered college entrance • Includes solos, etudes, scales/arpeggios, ear training, sight reading, theory • Curricula online • Adapt for your program

39. Performing Ensembles

40. Excellence in Theory or Standard of Excellence Music Theory & History Workbooks • Kjos - publisher • 3 volumes (see handout sample) • Includes theory, ear training, history • Take MS & HS to complete 3 volumes • Students work on lessons during down time in rehearsal • Establish grade level expectations and written exam

41. Insuring Integrity

42. Insuring Integrity • Self created, administered, and graded assessments • Colleagues & administrators will ask • Standards Based assessments • Comparable across classrooms • Demonstrate validity & reliability • Explain/demonstrate process for creating, administering, & grading • Demonstrate connection b/w state standards and assessments • Archive recordings

43. www.vocaroo.com • Audio emails • Archived up to 5 months • Sends link to an email address • Download as .WAV or .Ogg • Useful for performance tests • Very easy! • http://vocaroo.com/?media=vAdx5RJr1DVC7upIc

44. Festival Ratings

45. NAfME Position Statement • Successful music teacher evaluation must, where the most easily observable outcomes of student learning in music are customarily measured in a collective manner (e.g., adjudicated ratings of large ensemble performances), limit the use of these data to valid and reliable measures and should form only part of a teacher’s evaluation. (NAfME, 2011)

46. Festival Ratings: Advantages • Provide quantitative third party assessment • Can show growth over time in some circumstances • Individual judges’ ratings • Repertoire difficulty • 3 yr. period • Valid to the extent that they measure the quality of an ensemble’s performance of three selected pieces & sight reading at one point in time • Likely reliable over 3-yr. period based on previous research • Probably adaptable to state-wide evaluation tool • Assess a few performance standards

47. Ratings Growth Example Hypothetical Contest Ratings for One Ensemble over a Three-year Period Note. Roman numerals represent division ratings. a Total increase from year 1 to year 3 = 44%.

48. Ratings ≠ MEAP or MME Exams MEAP & MME • Same for all each yr. • Rel. and val. established • Many Standards • Individual • Mostly objective • Reflect multiple levels of achievement Ratings • Rep., adj. change • Val. & rel. not est. • Per. standards only • Group • Mostly subjective • 90%+ earn I or II out of V ratings.

49. Festival/Contest Ratings: Challenges • Reliability • Curricular limitations • Score Inflation • Ratings Effectiveness in differentiating quality • Influence of non-performance factors • Group vs. Individual performance • Other factors • Role of MSBOA & MSVMA?

50. Experiences