Release of 2011-12 Preliminary Value-Added Data Webinar August 13, 2012

1. Release of 2011-12 Preliminary Value-Added Data Webinar August 13, 2012 Florida Department of Education

2. Topics to Discuss • Review of state-level results of the model • Review of data files received on August 6, 2012 • Discussion of the data provided for non-FCAT subject area teachers and guidelines for the use of these data • Question and Answer session Florida Department of Education

3. Review of State-Level Results The next series of slides will review: • The model fit in 2012 compared to 2011 • Comparison of the Reading and Mathematics VAM scores by Grade • Impact analyses showing relationship of VAM score with various classroom characteristics Data analyses presented on slides 4-15 provided by American Institutes for Research (AIR) Florida Department of Education

4. Model Fit: Comparison of R-Square Values for Reading The R-Square values – which are an indicator of the how well the model fits the data – are high and virtually equivalent to the values observed with the 2011 model The model behaves the same using FCAT 2.0 data, as it did using FCAT data

5. Model Fit: Comparison of R-Square Values for Math The R-Square values – which are an indicator of the how well the model fits the data – are high and virtually equivalent to the values observed with the 2011 model The model behaves the same using FCAT 2.0 data, as it did using FCAT data

6. Teacher VAM Score by Grade (Reading) VAM scores by grade show no particular advantage for any given grade

7. Teacher VAM Score by Grade (Math) VAM scores by grade show no particular advantage for any given grade

8. Impact Analyses for VAM Scores • The following slides show the relationship of the teacher VAM score with various classroom characteristics • The observed score correlations for each characteristic are reported after the scatter plots • In all cases, the correlations are negligible indicating no advantages or disadvantages for any group of students

9. Correlation of Teacher VAM Score and Percent Students with Disabilities

10. Correlation of Teacher VAM Score and Percent English Language Learners

11. Correlation of Teacher VAM Score and Percent Economically Disadvantaged

12. Correlation of Teacher VAM Score and Percent Gifted

13. Correlation of Teacher VAM Score and Percent Non-White Students

14. Correlation of Teacher VAM Score and Mean Prior Achievement

15. Observed Correlations with Teacher VAM Score

16. Review of Data Files Received on August 6 Files for FCAT Subject Area Teachers (Reading and Mathematics) • State-level file • District-level files • School-level files • Teacher-level files • Student-level files • Student-Teacher link files Florida Department of Education

17. Review of Data Files Received on August 6 • State-, district-, and school-level files provide summary statistics that many districts have used as the basis for their classification decisions • Teacher-level files provide the performance data for reading/language arts and mathematics teachers • Teacher_VAM_estimate is the value-added score • Teacher_VAM_estimate_SE is the standard error associated with the value-added score • Pct_Meet_Expectations is the percentage of the teacher’s students who met or exceeded their expectation as determined by the value-added model Florida Department of Education

18. Review of Data Files Received on August 6 • Student-Level files provide data on the students included in the analysis, including the predicted (_1112_predictedscore) and actual scores (_1112_scalescore) • Student-Teacher Link files provide the students who were attributed to each teacher in the calculation of his/her value-added score • This file should be used to ensure that the appropriate students are linked to the appropriate teachers • Only students for which necessary data (e.g., at least two years of assessment data) are available are included Florida Department of Education

19. Review of Data Files Received on August 6 Aggregate school- and teacher-level files were also provided • These files combine value-added data (which is run at the subject and grade level) across grades, subjects, and yearsusing the common metric of a proportion of an average year’s growth • The values for an average year’s growth were also provided to districts, as well, on August 6 • The value-added scores are converted into proportions and then combined • A request has been made to provide an aggregate file for only 2011-12 (i.e., does not combine across years). That file should be delivered to districts early this week Florida Department of Education

20. Review of Data Files Received on August 6 In addition to files relating to FCAT subject area teachers, files were provided attributing the Reading and Mathematics value-added results to non-FCAT subject area teachers • Teacher-level files for non-FCAT subject area teachers were provided • Student-Teacher link file for non-FCAT subject area teachers were provided Florida Department of Education

21. Data for Non-FCAT Subject Area Teachers and Guidelines for Use Section 1012.34(7)(e), Florida Statutes, requires that for classroom teachers of courses for which the district has not implemented appropriate assessments or for which the school district has not adopted an equally appropriate measure of student learning growth, student learning growth must be measured by the growth in learning of the classroom teacher’s students on statewide assessments Florida Department of Education

22. Data for Non-FCAT Subject Area Teachers and Guidelines for Use • To meet this requirement, student level Reading and Mathematics value-added results were attributed to non-FCAT subject area teachers serving students who took statewide assessments to calculate two measures that can be used within the evaluation of these teachers • The percentage of the teacher’s students meeting/exceeding expectations as determined by the value-added model • The average of the teacher’s students’ residuals (i.e., differences between the students’ actual and expected scores as determined by the value-added model) Florida Department of Education

23. Data for Non-FCAT Subject Area Teachers and Guidelines for Use • The average of student residuals is conceptually similar to a teacher’s value-added score • It is a measure reflecting, on average, the degree to which teacher’s students exceeded or missed their expected score as determined by the value-added model • This data can be used in a similar fashion as the value-added score to classify teacher performance Florida Department of Education

24. Data for Non-FCAT Subject Area Teachers and Guidelines for Use Suggested guidelines for the use of the “average of residuals” score • If using state- or district-level summary statistics (e.g., statewide mean) as a basis for classification, rely upon statewide (or district-wide) summary statistics for this group of teachers • Statewide and district-wide summary statistics will be provided to districts this week for this group of teachers • Use the standard error, as your local plans outline, to assist in classifying teacher performance • The August 6 files contained the standard deviation of the average of the residuals; the standard error can be calculated, as well, and will be provided to districts in updated data this week Florida Department of Education

25. Other Topics • Courses used in the calculation of the FCAT Reading and Mathematics value-added model • Two elementary reading/language arts courses that some school districts began reporting in 2011-12 were not included in the preliminary value-added analysis • Course 5010045 • Course 5010046 • These courses will be added during the reanalysis phase Florida Department of Education

26. Other Topics • Courses used in the calculation of the FCAT Reading and Mathematics value-added model • AP and IB language arts and mathematics courses have not been included in the FCAT value-added analyses since they are courses with their own assessments measuring the college-level/international standards of those courses • There has been a request to add these courses to the analysis in order to produce an FCAT value-added score for AP/IB teachers who teach students in language arts or mathematics taking FCAT • These courses will be added during the reanalysis phase Florida Department of Education

27. Other Topics • Since the predictions derived from value-added models are based on how students and classrooms perform statewide, given characteristics controlled for in the model, changes to the underlying data submitted through the review period necessitates a re-calculation of the model • Important, this will result in a new set of teacher value-added scores for all, based on the verified data • Given this fact, we advise that districts await final value-added results, following the review period, before finalizing evaluations Florida Department of Education

28. Wrap-Up • Reminder: Data review period is currently scheduled to close on Thursday, September 6 at 5:00 EDT • Final value-added data should be delivered by the end of September • We will likely extend the October 19 reporting deadline for evaluation results to the Department • Additional files to be delivered this week are: • One-year aggregate FCAT VAM file (2011-12) for teachers and schools • Updated non-FCAT teacher file with the standard error statistic for the “average of residuals” included • State- and district-level summary statistics for non-FCAT teachers on the “average of residuals” measure Florida Department of Education

29. Contact Information • Juan Copa • juan.copa@fldoe.org • (850) 245-0744 • Questions? Florida Department of Education