Rosalita Mitchell, University of New Mexico

1. New Mexico’s Three-Tiered Licensure System And Its Implications for Assessing Teacher Preparation Effectiveness Rosalita Mitchell, University of New Mexico Peter Winograd, New Mexico Office of Education Accountability Cindy Sims, Moriarty Municipal Schools James Ball, New Mexico Public Education Department Madeline Feijoo, New Mexico Office of Education Accountability Smith Frederick, University of New Mexico Wanda Trujillo, Office of Education Accountability Scott Hughes, Office of Education Accountability Beata Thorstensen, Office of Education Accountability Presented at the Conference on Quality Teacher Preparation: Assessment of Teacher Preparation Effectiveness. Alverno College, Milwaukee, Wisconsin, September 30 - October 1, 2005 Sponsored by the National Commission on Teaching and America’s Future and the Carnegie Corporation of New York

2. Presentation Overview • The Debates About Teacher Preparation • The Key Question: Are Teacher Preparation Programs Effective? • New Mexico’s Three-Tiered Licensure System • A Statewide Study of Teacher Preparation Program Graduates’ Performance in the Classroom • What Can We Learn From This Study About Assessing The Effectiveness of Teacher Preparation Programs? • Summary

3. The Debate About Teacher Preparation • The Critics - Those who can’t do, teach; Those who can’t teach, teach teachers: • Too many weak students enter the teaching profession • Too many graduates are poorly equipped to teach • Programs focus too much on "soft" pedagogical knowledge at the expense of subject-matter depth • Programs fail to prepare graduates to teach to student performance standards • Programs do not provide adequate, real-world, practical experience

4. The Debate About Teacher Preparation • The Response – Our programs are good and getting better: • NCTAF and other national initiatives • Federal Efforts (NCLB, Title II) • Teacher Preparation Improvements (school partnerships, more university support, P-20 systems, induction programs, alternative licensure programs) • Rigorous Accreditation Standards • NCATE - Standard 2: Assessment System and Unit Evaluation. NCATE (NCATE 2002, p. 10) defines Standard 2 as “The unit has an assessment system that collects and analyzes data on the applicant qualifications, candidate and graduate performance, and unit operations to evaluate and improve the unit and its programs

5. The Key Question: Do preparation programs prepare teachers to teach all students how to succeed academically and in other important ways? The Key Answer: Assess program graduates who are working with real children in actual classrooms

6. New Mexico’s Three-Tiered Licensure System “The Legislature further finds that the teacher shortage in this country has affected the ability of New Mexico to compete for the best teachers, and that unless the state and school districts find ways to mentor beginning teachers, intervene with teachers while they still show promise, improve the job satisfaction of quality teachers and elevate the teaching profession by shifting to a professional educator licensing and salary system, public schools will be unable to recruit and retain the highest quality teachers in the teaching profession in New Mexico.”HB 212 – The Public School Reforms Act, 2003“If our future generations are to prosper, then they must have the skills to compete with the best – and brightest – in the world. In education, the classroom is where the rubber meets the road. Our teachers are the focus of everything that happens in education…” Governor Bill Richardson, State of the State Address, January 21, 2003

7. An Overview of Key Elements of NM’s 3-Tiered Licensure System $30K Minimum $40K Minimum $45K in 06-07; $50K in 07-08 Level III Master Teacher Level I Provisional Teacher Level II Professional Teacher √ Successful Annual Evaluations at Level II Indicators; √ Verification Teacher is Highly Qualified if assigned to a core content area; √ Licensure Renewal; √ Successful Annual Evaluations at Level I Indicators; √ Mentoring; √ Verification Teacher is Highly Qualified if assigned to a core content area; √ Successful Annual Evaluations at Level III Indicators; √ Verification Teacher is Highly Qualified if assigned to a core content area; √ Licensure Renewal; • Advance to Level III by • submitting Professional • Development Dossier (PDD); • Minimum 3 Years Level II • Teaching Experience; • MA or NBPTS • Advance to Level II by • submitting Professional • Development Dossier (PDD); • Minimum 3 Years Level I • Teaching Experience

8. What is The Professional Development Dossier (PDD)? • A performance-based assessment centered on nine NM Teacher Competencies • A focused, compact collection of documentation compiled by a NM teacher for licensure advancement • Structured documentation of student achievement and student learning

9. Defining Teacher Quality…Finding a Balance Teacher performance and student learning Student Teacher’s content test scores knowledge NM Teacher Evaluation System

10. New Mexico’s Nine Teacher Competencies • The teacher accurately demonstrates knowledge of the content area and approved curriculum. • The teacher appropriately utilizes a variety of teaching methods and resources for each area taught. • The teacher communicates with and obtains feedback from students in a manner that enhances student learning and understanding. • The teacher comprehends the principles of student growth, development and learning and applies them appropriately. • The teacher effectively utilizes student assessment techniques and procedures. • The teacher manages the educational setting in a manner that promotes positive student behavior and a safe and healthy environment. • The teacher recognizes student diversity and creates an atmosphere conducive to the promotion of positive student involvement and self-concept. • The teacher demonstrates a willingness to examine and implement change, as appropriate. • The teacher works productively with colleagues, parents and community members.

11. The Three-Tiered Licensure System Organized New Mexico’s Nine Teacher Competencies Into Three Strands • Strand A – Instruction 1. The teacher accurately demonstrates knowledge of the content area and approved curriculum. 2. The teacher appropriately utilizes a variety of teaching methods and resources for each area taught. 5. The teacher effectively utilizes student assessment techniques and procedures. • Strand B – Student Learning 3. The teacher communicates with and obtains feedback from students in a manner that enhances student learning and understanding. 4. The teacher comprehends the principles of student growth, development and learning and applies them appropriately. 6. The teacher manages the educational setting in a manner that promotes positive student behavior and a safe and healthy environment. 7. The teacher recognizes student diversity and creates an atmosphere conducive to the promotion of positive student involvement and self-concept. • Strand C – Professional Growth 8. The teacher demonstrates a willingness to examine and implement change, as appropriate. 9. The teacher works productively with colleagues, parents and community members.

12. The Three-Tiered Licensure System Provided More Depth To Each Of The Nine Competencies For example:

13. Review of PDD Design Strand A -- Instruction (Independent Review) Strand B -- Student Learning (Independent Review) Strand C -- Professional Growth (Independent Review) Strand D -- Verification of teaching experience, mentoring, and authenticity Strand E -- Superintendent’s recommendation for advancement based on annual evaluations

14. A Statewide Study of Teacher Preparation Program Graduates’ Performance in the Classroom: Overview • 997 Level 1 and 2 teachers submitted a Professional Development Dossier (PDD) as part of New Mexico’s Three-Tiered Licensure System during the January-February, 2005 Submission Period • Teachers could receive three kinds of scores: • Did not pass strands A, B, C – (146 teachers or 14.6%) • Did not receive school administrator approval (16 teachers or 1.6%) • Pass (835 teachers or 83.8%)

16. 15

17. Percent of All Level 1 and 2 Teachers Passing By Gender(N=997) Note: (Number of Teachers In Each Gender Category)

18. Percent of All Level 1 and 2 Teachers Passing By Ethnicity (N=651) Note: (Number of Teachers In Each Ethnic Category)

19. Percent of All Level 1 and 2 Teachers Passing By Type of License (N=843) Note: (Number of Teachers In Each Type of License)

20. Evaluating New Mexico Preparation Program Graduates’ Performance in the Classroom • 701 of the 997 teachers were Level 1 teachers attempting to advance to Level 2 AND had been prepared in New Mexico teacher preparation programs • These 701 Level 1 teachers prepared in NM programs could receive three kinds of scores: • Did not pass strands A, B, C - (93 teachers or 13.3%) • Did not receive school administrator approval (14 teachers or 2.0%) • Pass (592 teachers or 84.7%) • Detailed information about which competencies teachers missed were available on 42 of the 93 teachers who did not pass strands A.B, or C

21. Preliminary Analyses For Level 1 Teachers Prepared in New Mexico Programs • How did the passage rate for Level 1 teachers vary by New Mexico teacher preparation program? • How did the passage rate for Level 1 teachers vary by ethnicity? • How did the passage rate for Level 1 teachers vary by district? • How did the passage rate for Level 1 teachers vary by program type? • Which of the nine teacher competencies proved most difficult for those Level 1 teachers who did not pass? • Which of the nine teacher competencies proved most difficult for Level 1 elementary, secondary and special education teachers who did not pass?

22. Percent of Teachers Passing From Level 1 To Level 2 By Preparation Program (In State N=697; Out of State N = 226)) Note: (Number of Teachers for Each Preparation Program)

23. Percent of Teachers Passing From Level 1 To Level 2 By Ethnicity (N=461) Note: (Number of Teachers In Each Ethnic Category)

24. Percent of Teachers Passing From Level 1 To Level 2 By Type of Preparation Program (Alternative Licensure or Traditional Licensure) (N=698) Note: (Number of Teachers In Each Type of Program)

26. Which competencies proved most difficult for those Level 1 teachers who did not pass?

27. Which Competencies Proved Most Difficult For Elementary, Secondary And Special Education Level 1 Teachers Who Did Not Pass?

28. What Can We Learn From This Study About Assessing The Effectiveness of Teacher Preparation Programs?

29. How Might Preparation Programs And Districts Use This Data? • Consider the pass rates and competency analyses as a valid sources of information. • Avoid the blame game, the “good enough” syndrome, or it’s “not our responsibility” reflex. • Think systematically about the skills, knowledge, and dispositions that students bring to their college experiences, including their time in teacher preparation programs. • Think systemically about the 1-2 years of teacher preparation and the 1-3 years of district mentoring and professional development. • Examine how the competencies used for preparation programs align with the 9 competencies used to evaluate teachers. • Examine how the district’s mentoring and professional development programs align with the 9 competencies used to evaluate teachers. • Involve college and school faculty in strengthening the preparation and support of beginning teachers.

30. For Example: Examine the Alignment Between the Competencies Used In Teacher Preparation Programs And Teacher Evaluation

31. For Example: Conduct Detailed Analyses Who Passed and Failed and Why By Specific Programs and Districts

32. For Example: Examine The Issues in The District’s Professional Development Program • Teachers need opportunities to reflect on the effectiveness of instructional practice in advancing student learning. • Teachers need opportunities to identify relevant student learning data, understand what the data is showing for individual students as well as the whole class in terms of gaps in the instructional program and strategies needed to address those gaps. • Competency 4 requires evidence of instructional differentiation to address individual students’ needs. Teachers need professional development to address student learning levels, rates and styles using appropriate strategies and resources. • Currently, school districts provide significant support for level 1 teachers to successfully complete dossier to advance to a level 2 status. Professional development opportunities should provide the same level of support to level 2 teachers in their pursuit of a level 3 status.

33. What are the implications of this study for assessing teacher preparation effectiveness? • We can develop and implement systems for assessing teacher quality based on a balance of the teacher’s background and knowledge AND student growth. • Assessments of teacher quality can identify the areas that teachers find particularly challenging. • It is important to remember that beginning teachers spent 1-2 years in their preparation program and 3 years in their school district before submitting their materials to the Professional Development Dossier. • We must think carefully about how to assess the performance of program graduates over time – their strengths and areas for improvement may change from year to year. • We must think carefully about what can be attributed to a preparation program and what should be attributed to district’s training and support.

34. Summary • Teacher preparation programs should prove their effectiveness by gathering data about the performance of their graduates. • Gathering almost any data is a start because data will show what needs to be done. • We must think carefully about what can be attributed to a preparation program and what should be attributed to district’s training and support. • We must work collaboratively: • For the State: Increased understanding of areas for teaching improvement. • For University Faculty: Increased responsiveness to the needs of program graduates in the field. • For School Districts: Improved professional development focused on student learning.