Jeanne M. Burns, Ph.D. Louisiana Board of Regents & Office of the Governor

1. LOUISIANA TEACHER PREPARATION ASSESSMENT MODEL Jeanne M. Burns, Ph.D. Louisiana Board of Regents & Office of the Governor University of Wisconsin System 2007 Quality Matters Conference November 10, 2007

2. MAJOR ACCOMPLISHMENT Connected Foundation for Teacher Preparation Program Success 2006-2007 Partially Connected Foundation for Teacher Preparation Program Success 2000-2006

3. Louisiana’s Blue Ribbon Commission for Educational Excellence The successful PK-16+ state advisory group that has lead Louisiana’s efforts to improve the quality of teachers and has supported the implementation of recommendations across multiple administrations from 1999-Present. Foster Administration 1999-2004 Blanco Administration 2004-Present Future

4. Louisiana’s Teacher Preparations Programs are Currently Meeting State and National Standards

5. Standards in Louisiana & Other StatesNew Redesigned Teacher Preparation Programs Success in Louisiana 100% 0% 100% 0%

6. Standards in Louisiana & Other StatesPassage Rates on Examinations for Teachers 99% 99% Success in Louisiana 99% 99% 96% 90% 89%

7. Standards in Louisiana & Other StatesProduction of New Teachers by Teacher Preparation Programs 2,374 (New Baseline) Hurricane Katrina & Hurricane Rita Success in Louisiana 2,727 2,664 2,659 2,336 2,589

8. Standards in Louisiana & Other StatesTeachers in Public Schools with Standard Teaching Certificates 95.34% 93.79% Success in Louisiana 90.8% 88.10% 84,38% 84.85% 85.77%

9. Standards in Louisiana & Other StatesNationally Accredited Teacher Preparation Programs Success in Louisiana 100% 93% 100% 80% 20%

10. OtherMeasures of Success • Louisiana Teacher Preparation Accountability System - 17 out of 19 public and private universities attained labels of High Performing or Exemplary when the system was last implemented and received monetary rewards from the Board of Regents • Education Week Quality Counts Report for 2005 & 2006 – Louisiana Received a Grade of A in Efforts to Improve Teacher Quality • U.S. Chamber of Commerce Education Report Card – Louisiana received a Grade of A for 21st Century Teaching Force in 2007 Success in Louisiana

11. Louisiana’s Teacher Preparation Programs are Now Taking a Bold Step Forward to Surpass Standards in Other States toExamine Growth of Learning of Children Taught by Graduates Louisiana will be the first state in the nation to examine this level of effectiveness on a statewide basis.

12. Concerns About Teacher Preparation in 1999-2000 • Low passage rates on PRAXIS examinations for teachers within some teacher preparation programs (e.g., 33%, 38%, 65%). • Low number of teachers completing teacher preparation programs in math, science, & special education. • Low number of minorities completing teacher preparation programs. • Perception that teachers were not satisfied with their teacher preparation programs. • High percentage of uncertified teachers.

13. Board of Elementary & Secondary Education – New PoliciesTeacher Certification New Certification Structures • PK-3, 1-5, 4-8, 6-12 New Certification Structures (Standards Based) • Regular Undergraduate • Alternate Certification - Master of Arts in Teaching (36 credit hours) - Practitioner Teacher Program (21-30 credit hours) - Non-Master’s/Certification-Only Program (24-33 credit hours) Core Components of All Programs • Content • Focus Area • Knowledge of the Learner & Learning Environment • Methodology • Student Teaching/Internship New Praxis Examinations & Higher Cut-off Scores

14. Board of Regents – New Policies Master Plan for Post-Secondary Education STRATEGIES FOR TEACHER PREPARATION • Expand recruitment efforts for education majors. • Create coordinated partnerships between • universities and local school systems. • Revamp existing teacher education • programs. • Design and implement alternative • certification programs. • Expand financial aid programs • for education majors.

15. Louisiana’s Teacher Preparation Programs: Four Levels of Effectiveness Level 4: Effectiveness of Growth in Student Learning (Value-Added Teacher Preparation Program Assessment) Level 3: Effectiveness of Impact (Teacher Preparation Accountability System) Level 2: Effectiveness of Implementation (NCATE – Comprehensive Assessment System) Level 1: Effectiveness of Planning (Redesign of Teacher Preparation Programs)

16. EFFECTIVENESS OF TEACHER PREPARATION PROGRAMS: LEVEL 1PLANNING (REDESIGN OF PROGRAMS) • Redesign Purpose: To align all university programs with state and national PK-12 content standards, state and national PK-12 standards for teachers, PRAXIS, and NCATE. • Redesign Period: October 2001 to June 2003. • Redesign Teams: Colleges of Arts/Sciences, Colleges of Education, and school/district personnel. • Redesign Evaluators: External national experts. • University Development: Redesign for Educational Excellence Institutes. • Deadline for Redesign Program Approval: July 1, 2003.

17. EFFECTIVENESS OF TEACHER PREPARATION PROGRAMS: LEVEL 2IMPLEMENTATION (NCATE & Assessment System) • State Expectation: NCATE Accreditation for all public and private institutions. • Assessment System: Systematic assessment of knowledge, skills, and dispositions of teacher candidates.

18. EFFECTIVENESS OF TEACHER PREPARATION PROGRAMS: LEVEL 3IMPACT (TEACHER PREPARATION ACCOUNTABILITY SYSTEM) • Institutional Performance Index (Weight = 50%) - PRAXIS Passage Rates of Program Completers - Ratings of First Year Teachers on Satisfaction Survey (Ratings of Mentors of First Year Teachers on Satisfaction Survey) • Quantity Index (Weight = 50%) • School and District Partnership Index (Future)

19. First Year Teacher Satisfaction Survey Structure: • Aligned with Louisiana’s Standards for Teachers (Louisiana Components of Effective Teaching) • 33 Items on Survey – 4 Point Rating Scale • Areas Addressed: Planning, Instruction, Management, & School Improvement Sample Item and Overall Mean Scores: 11. Successfully plan for individual student differences in the context of a whole class. 2001-2002 Mean Score 2002-2003 Mean Score 2.99 3.20

20. EFFECTIVENESS OF TEACHER PREPARATION PROGRAMS: LEVEL 3IMPACT (TEACHER PREPARATION ACCOUNTABILITY SYSTEM) • Institutional Performance Index (Weight = 50%) • Quantity Index (Weight = 50%) - Total Number of Program Completers - Number of Minority Program Completers - Number of Program Completers in Teaching Shortage Areas (e.g., Math, Science, Special Education, & Middle School) - Number of Male-Elementary & Male-Early Childhood Program Completers - Number of Program Completers Accepting Teaching Positions in School Districts with the Highest Percentage of Uncertified Teachers • University/District Partnership Index – To be called “Growth of Student Learning Index” in the future. LABELS, REWARDS, & CORRECTIVE ACTIONS

21. Number of Completers in Teacher Shortage Areas for Mathematics 127 119 85 28

22. Number of Program Completers in Teacher Shortage Areas for Special Education 334 257 199 67

23. Louisiana’s Teacher Preparation Programs: Four Levels of Effectiveness Level 4: Effectiveness of Growth in Student Learning (Value-Added Teacher Preparation Program Assessment) Level 3: Effectiveness of Impact (Teacher Preparation Accountability System) Level 2: Effectiveness of Implementation (NCATE – Comprehensive Assessment System) Level 1: Effectiveness of Planning (Redesign of Teacher Preparation Programs)

24. Technical Report: Value Added Assessment of Teacher Preparation (2006-07) George H. Noell, Ph.D. Department of Psychology Louisiana State University

25. Acknowledgement of Key People Who Have Made This Research Possible • Commissioner of Higher Education E. Joseph Savoie, Past State Superintendent Cecil Picard, Current State Superintendent Paul Pastorek • Louisiana Department of Education Division of Planning, Analysis, and Information Resources (David Elder – Division Director, Allen Schulenberg, Robert Kaufman, Kelvin LaCroix, Steve Gunning, Sam Pernici, and Roth Aymond) • LSU Research Team (Veronica Gulley, Bethany Porter, Anna Beth Ball, Maria Patt, Amanda Dahir)

26. 2006-07 StudyBreakdown of Data • Years 2004-2005 and 2005-2006 • Student Grade Levels: Grades 4-9 • Content Areas: Mathematics; Science; Social Studies • Tests:Iowa Test of Basic Skills, i-LEAP, and LEAP-21 • Pathways for New Teachers: Undergraduate and Alternate Certification Programs

27. NEW AND EXPERIENCED TEACHERS • New Teachers: • 1st and 2nd year teachers with regular certificates • Completed Teacher Preparation Program within 5 years • Teaching within area of certification • Experienced Teachers • 3rd or subsequent year teachers with regular certificates • Teaching within area of certification

28. Criteria for Inclusion of New Teachers in Study • Inclusion for each content area • 25 or more new teachers in grades 4-9 • At least 10 new teachers per year • Teaching within certification • Remained with student full academic year • Not all universities had sufficient new teachers to be included in the analysis. Another year of data will be added and will increase the number that exceed 25 graduates.

29. Size of the Data Basefor the Study • Districts: All school districts in Louisiana • Students: 285,000+ • Teachers: 7,000+ • Schools: 1,300+ • Data Linkages: 9,000,000+ data linkages

30. Value-Added ModelProcess • Predict student achievement • based on prior achievement, demographics, and attendance. • Assess actual student achievement. • Calculate degree to which students taught by new teachers met achievement of similar students taught by experienced teachers.

31. Value-Added Model Predictors:Hierarchical Linear Models

32. Teacher Preparation Effect Estimates • Based on at least 25 new teachers per program across multiple school districts in Louisiana. • Reflects a pattern of effectiveness of new teachers based on the average difference between the achievement of students taught by new teachers and experienced teachers.

33. Post-Redesign Programs & Pre-Redesign Programs • Post-Redesign Programs: Programs that were redesigned for grades PK-3, 1-5, 4-8, and 6-12 and began admitting pre-service teachers on July 1, 2003. • Pre-Redesign Programs: Universities stopped admitting candidates to the programs on July 1, 2003. A phase out period is occurring for pre-redesign programs.

34. Use of Effect Estimates • Effect estimates for post-redesign programs will be used to examine the effectiveness of the post-redesign programs. • Effect estimates for pre-redesign programs will be used as benchmarks to compare the new and old programs once data are available for the post-redesign programs. Candidates are no longer admitted to the pre-redesign programs, and the programs no longer reflect the full content of the post-redesign programs.

35. Performance Levels Analysis and Performance Bands • Level 1: Programs in which new teachers are more effective than experienced teachers. • Level 2: Programs in which new teachers are comparable in effectiveness to experienced teachers. • Level 3: Programs in which new teachers are comparable in effectiveness to new teachers. • Level 4: Programs in which new teachers are less effective than new teachers. • Level 5: Programs in which new teachers are statistically significantly less effective.

36. POST-REDESIGN PROGRAMS

37. Post-Redesign Programs2006-07 Study Results • Alternate Certification Louisiana College Northwestern State University The New Teacher Project Alternate certification programs were redesigned before undergraduate programs. Students were admitted to the alternate certification programs at an earlier date and could complete the programs in a shorter time period since they already possessed a baccalaureate degree. Thus, data for post-redesign undergraduate programs are not yet available. It is anticipated that data will be available for large post-redesign undergraduate programs next year.

38. Effect Estimates for Post-RedesignLouisiana CollegeAlternate Certification Program

39. Effect Estimates for Post-RedesignNorthwestern State UniversityAlternate Certification Program

40. Effect Estimates for Post-RedesignThe New Teacher ProjectAlternate Certification Program Fewer than minimum number of new teachers. Fewer than minimum number of new teachers.

41. Placing Results in Context: Mathematics Experienced Teachers

42. PRE-REDESIGN PROGRAMS Universities admitted students to the programs prior to July 1, 2003.

43. Number of Universities at Different Performance LevelsPre-Redesign Alternate Certification Programs

44. Number of Universities at Different Performance LevelsPre-Redesign Undergraduate Programs

45. Major Findings • It is possible to implement a system that measures the effectiveness of teacher preparation programs based upon the achievement of students taught by new teachers who graduated from those teacher preparation programs. • It is possible for teacher preparation programs to prepare new teachers whose students demonstrate achievement that is comparable to the achievement of students taught by experienced teachers.

46. Next Steps • Teacher Preparation Accountability System - Expand existing accountability system to include Institutional Performance Index, Quantity Index, and Growth of Student Learning Index (effect estimates) • State Research Team (Grant from the Carnegie Corporation of New York) - Work with state team composed of one researcher from every teacher preparation program to collect consistent data across campuses to answer questions pertaining to “why” new teachers from some teacher preparation programs have higher achieving students than others. • National Research Council - Work with 4 other states to compare value -added results across states.

47. FOR ADDITIONAL INFORMATION George Noell gnoell@lsu.edu Jeanne M. Burns jeanne.burns@la.gov We’ve come a very, very, very, long way! 1999-2007 http://www.regents.state.la.us/Academic/TE/Value%20Added.htm

48. QUESTIONS • What questions do we need to be asking to understand why some teacher preparation programs are producing new teachers whose students demonstrate growth in achievement that is comparable to growth in achievement of children taught by experienced teachers? • What data can be collected to answer the questions?