1 / 24

VALUE-ADDED MODELS AND THE MEASUREMENT OF TEACHER QUALITY

VALUE-ADDED MODELS AND THE MEASUREMENT OF TEACHER QUALITY. Douglas Harris Tim R. Sass Dept. of Ed. Leadership Dept. of Economics and Policy Studies Florida State University Florida State University (tsass@fsu.edu) (harris@coe.fsu.edu). IES Research Conference – June 2006.

arlenebaker
Download Presentation

VALUE-ADDED MODELS AND THE MEASUREMENT OF TEACHER QUALITY

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. VALUE-ADDED MODELSAND THE MEASUREMENT OF TEACHER QUALITY Douglas Harris Tim R. Sass Dept. of Ed. Leadership Dept. of Economics and Policy Studies Florida State University Florida State University (tsass@fsu.edu) (harris@coe.fsu.edu) IES Research Conference – June 2006 -- Preliminary, Do Not Quote Without Permission --

  2. Evaluating Value-Added Methodology • The recent availability of panel data has produced a flood of research studies using various “value-added” approaches • Research Questions • Are assumptions underlying the value-added approach valid? • Are some methods more likely to produce reliable estimates than others? • What data are most important to obtaining consistent estimates? -- Preliminary, Do Not Quote Without Permission --

  3. Evaluating Value-Added Methodology • Basic Model Types • Cumulative Model • Unrestricted Value-Added Model • Value-Added Models with Persistence Restrictions • Restricted Value-Added or “Gain-Score” Model • Contemporaneous Model • Specification Issues for Value-Added Models • Treatment of teacher heterogeneity • Measures of classroom/school inputs • Treatment of student heterogeneity • Aggregation -- Preliminary, Do Not Quote Without Permission --

  4. General Cumulative Model of Student Achievement -- Preliminary, Do Not Quote Without Permission --

  5. Basic Assumptionsof Value-Added Models • Cumulative achievement function does not vary with age and is additively separable. • Family inputs are time invariant. • Parents do not compensate for poor school inputs or poor outcomes • Todd and Wolpin (2005) reject exogeneity of parental inputs at 90 percent, but not at 95 percent confidence level • The marginal inputs of all school-based inputs, parental inputs, and the initial student endowment each decline geometrically (at potentially different rates) over time. • Lagged achievement serves as a sufficient statistic for prior inputs • We find twice-lagged inputs do not provide additional information -- Preliminary, Do Not Quote Without Permission --

  6. UnrestrictedValue-Added Model -- Preliminary, Do Not Quote Without Permission --

  7. Persistence Restrictions • Restricted Value-Added or “Gain-Score” Model • l is assumed to equal 1 (no decay in effect of past inputs) • Alternatively, can interpret as an achievement growth model where growth is independent of past school inputs. • Contemporaneous Model • l is assumed to equal 0 (complete decay in effect of past inputs). -- Preliminary, Do Not Quote Without Permission --

  8. Decomposition of School-based Inputs in Value-Added Model -- Preliminary, Do Not Quote Without Permission --

  9. Modeling Teacher Heterogeneity • Substituting teacher time-invariant measured characteristics for teacher fixed effects -- Preliminary, Do Not Quote Without Permission --

  10. Classroom and School Inputs • Exclusion of peer variables (P-ijmt) • Number of peers (class size) and peer characteristics (gender, race, mobility, age) • If peer variables are correlated with student and teacher characteristics (Xit and Tkt), omission will produce inconsistent estimates • Exclusion of school fixed effects (fm) • Given that teachers do not frequently change schools, omission of school effects will mean that teacher fixed effects will capture both teacher effects and some of the school effect, leading to inconsistent estimates -- Preliminary, Do Not Quote Without Permission --

  11. Modeling Student Heterogeneity • Substituting measured time-invariant student characteristics for student fixed effects • Race/ethnicity, foreign/native born, language parent speak at home, free-lunch status • As with teachers, if unmeasured time-invariant student characteristics are correlated with independent variables, will get inconsistent estimates -- Preliminary, Do Not Quote Without Permission --

  12. Modeling Student Heterogeneity • Fixed vs. random student effects • Fixed effects allow for a separate intercept parameter for each student (equal to the mean error for that student) whereas random effects assume that the student-specific intercepts are drawn from a known distribution (typically normal) • Since random effects are part of the error structure, they must be orthogonal to the model variables (Xit, P-ijmt, Tkt) in order to yield consistent estimates • Given that fixed effects estimates are always consistent (whether or not unobserved student heterogeneity is correlated with other variables in the model), can test orthogonality assumption by applying a Hausman test • Multilevel fixed effects models have been computationally burdensome -- Preliminary, Do Not Quote Without Permission --

  13. Aggregation • Measuring characteristics of specific teachers vs. grade-level-within-school averages • Since Texas data does not identify specific teacher, work by Rivkin, Hanushek and Kain (2005) relies on average characteristics of teachers within a grade • Advantages/Disadvantages of aggregation • Eliminates problems associated with non-random assignment of students to teachers within a school • May reduce problem of measurement error since individual errors may cancel out at grade level • May upwardly bias estimated impacts of school resources in the presence of omitted variables • Tends to reduce precision of estimates -- Preliminary, Do Not Quote Without Permission --

  14. Data • Florida’s K-20 Education Data Warehouse • Census of all children attending public schools in Florida • Student records linked over time • Covers 1995/1996 – 2003/2004 school years • Includes student test scores and student demographic data, plus enrollment, attendance, disciplinary actions and participation in special education and limited English proficiency programs • Includes all employee records including individual teacher characteristics and means of linking students and teachers to specific classrooms -- Preliminary, Do Not Quote Without Permission --

  15. Sample for Analysis • Middle school students (grades 6-8) who took SSS-NRT (Stanford-9) math test in three consecutive years during 1999/2000 – 2003/2004 • Enrolled in a single math course in the Fall • Up to 4 years of achievement gains • 4 cohorts of students • Includes a variety of math courses, from remedial to advanced and gifted classes • Use random sample of 100 middle schools • Reduces computational burden of estimating fixed effects • Represents about 12% of middle schools in state -- Preliminary, Do Not Quote Without Permission --

  16. Value-Added Model EstimatesWith Varying Degrees of Persistence -- Preliminary, Do Not Quote Without Permission --

  17. Correlation of Estimated Teacher Effects From Models with Varying Degrees of Persistence -- Preliminary, Do Not Quote Without Permission --

  18. Restricted Value-Added Model EstimatesWith Differing Controls for Teacher Heterogeneity -- Preliminary, Do Not Quote Without Permission --

  19. Restricted Value-Added Model EstimatesWith Differing Classroom/School Controls -- Preliminary, Do Not Quote Without Permission --

  20. Correlation of Estimated Teacher Effects From Models with Differing Classroom/School Controls -- Preliminary, Do Not Quote Without Permission --

  21. Restricted Value-Added Model Estimateswith Differing Controls for Student Heterogeneity -- Preliminary, Do Not Quote Without Permission --

  22. Correlation of Estimated Teacher Effects From Models With Differing Controls for Student Heterogeneity -- Preliminary, Do Not Quote Without Permission --

  23. Restricted Value-Added Model Estimates --Teacher-Specific vs. Within-School Grade-Level Averages -- Preliminary, Do Not Quote Without Permission --

  24. Summary of Findings • Model Selection • Restricted value-added model seems to be a good approximation of the full cumulative model • Specification • Use of student and teacher fixed effects (rather than covariates) important • Random effects may yield inconsistent estimates • Important to include school fixed effects, but classroom peer variables relatively unimportant • Aggregation to the grade level has some effect, though estimates not radically different from estimates with teacher-level data -- Preliminary, Do Not Quote Without Permission --

More Related