Studying the Relationships Between and Among Teacher Education, Teacher Quality, and Pupil Learning: Strategies, Challen

1. Studying the Relationships Between and Among Teacher Education, Teacher Quality, and Pupil Learning: Strategies, Challenges, and Rewards Robert Tobias, Director Center for Research on Teaching and Learning NYU’s Steinhardt School of Culture, Education, and Human Development AACTE Annual Meeting February 25, 2007

2. STRATEGIES

3. Conceptual Framework for CRTL Research on the Steinhardt School’s Teacher Education Programs Causal Factors Steinhardt Courses MAP Courses Field Observations Student Teaching Extra-Curricular Experiences Advisement Independent Study Post-Graduate Study In-Service Prof. Dev. Prof. Experiences School Context Demographics Individuality Family Society Peers School Professional Educator Steinhardt Graduate Entering Steinhardt Student Birth – 12 Students • Claims: • Content Knowledge • Pedagogical • Knowledge • Teaching Skill • Teacher Caring/ • Efficacy Continued Professional Growth Through Reflection in Practice Outcomes Beliefs about : Teaching Efficacy Teacher Caring Self-Efficacy Content Knowledge Learning

4. Teacher Education Student Demographics and Pre-Program achievement data Grade Point Averages Content Area Pedagogical Knowledge Teaching Skill Domain-Referenced Student Teacher Observation Scale (DRSTO-R) Educational Beliefs Questionnaire (EBQ) NYS Teacher Certification Exam Scores (NYSTCE) Student Course Reaction Forms (SCRF) End-of-Term Student Teacher Feedback Questionnaires (ETFQ) Fast Track End-of-Program Questionnaire (FTEPQ) Undergraduate End-of-Program Questionnaire (UEPQ) State Information System Follow-Up Tracking Data One-Year Self-Report Follow-Up Survey One-Year Employer Follow-Up Survey Pupil Achievement Data Pupil Work Samples CRTL is Building a Comprehensive Database of Pre-Professional, Induction, and Follow-Up Measures

5. CRTL is Beginning the Fourth Year of the Ongoing Study of Steinhardt’s Teacher Education Programs • Extracting data from extant databases • Designing and validating instruments for measuring the developing expertise and dispositions of pre-service and in-service teachers. • Building a relational database (Phoenix) • Developing processes and methods for tracking and collecting data on graduates • Analyzing and reporting the data to inform continuous program improvement.

6. Follow –Up Study Phase 1 • Use state- and local-education agency (SEA and LEA) teacher databases to locate Steinhardt graduates. • Link SEA and LEA data with pre-service data in Phoenix and school characteristics data from LEA databases. Phase 2 • After obtaining approval from school administrators and graduates, administer a self-report survey of teaching experiences to graduates and a survey of graduates’ teaching expertise to principals. • Obtain standardized achievement test scores and work samples for graduates’ pupils. • Use HLM analytic techniques to assess the causal relationships among the variables for pre-service program experiences, teacher quality and disposition, and pupil achievement. The analysis will explore the interactive effects of school environment and pupil and teacher demographics.

7. Phase 1 Results • Identified 879 Steinhardt graduates from the Classes of 2001-2004 who were teaching in New York State public schools in October 2004 (BS=205; MA=674). Graduates were teaching in 546 public schools in 19 NYS counties; largest number in NYC (81.6%), with most in Manhattan (N=308), followed by Brooklyn (N=180), and the Bronx (N=105). • Large numbers of graduates were teaching in inner city schools serving mostly poor African American and Latino students, some with large numbers of English language learners and recent immigrants. • Most BS graduates were inducted into teaching directly after graduation and were continuously employed in the same schools and districts. • Science education majors had the highest employment rate (71%) followed by Literacy, Math, and Social Studies (over 50%) • Some schools had high concentrations of graduates with as many as 10 members of the cohort among the faculty. • Additional graduates were identified in Connecticut and Florida, as well as through a match to the October 2003 NYS database.

8. Phase 2 Status • Developed survey instruments. • Adapted standards-based pupil work assignments, grades 3-12. • Obtained IRB approvals. • Identified preliminary target sample of graduates. • Collaborating with the NYCDOE to update information on the teaching assignments of an augmented file of graduates. • Collaborating with the NYCDOE to obtain standardized test scores for graduates teaching in grades 4 – 8. • Adjusting the analytic growth model to the exigencies of the available data.

9. Challenges • Generalizability: Currently, tracking graduates is limited to public schools in NYS and Connecticut and, of course, participation in the study is voluntary. • Validity of the dependent measure: State and local testing programs were not designed for value-added research. • Validity and reliability of measures of teacher quality: License exam scores, GPAs, teacher observation scales, and self reports are all imperfect measures. • Specification of the model: Teaching and learning involves a complex interplay of a wide array of variables, many of which defy accurate measurement. • Limitations of data systems: Institutional data systems were designed for school management and not research. • Inter-organizational tension: The agendas of public school systems and IHEs may not be aligned. Collaborative partnerships are integral to reducing this tension.

10. Early Rewards • Accreditation • Identification of opportunities for school-IHE partnerships. • Leverage for the development of an evidence base for the evaluation of teacher education programs. • Support for a culture of data-based decision making. • Faculty discussions about the characteristics of high quality teachers and high quality teaching and learning show promise for increasing program coherence.