2004-2005 Performance-Based Monitoring Analysis System (PBMAS)

1. 2004-2005 Performance-Based Monitoring Analysis System (PBMAS) January 2005 Update

2. PBMAS 2004-05 District Reports • Reports were mailed to all district and charter school superintendents between December 13-15. • ESC packets (with reports for ESC’s districts) were also mailed to ESC executive directors during that time. • PBM Division is working to develop a “masked” version of the reports that can be accessed on line.

3. PBMAS 2004-05 District Reports (cont.) • No timeline yet for when “masked” versions of the reports might be available. ASAP is our goal. Please continue to check website: www.tea.state.tx.us/pbm. • 2004-2005 PBMAS Manual is available on-line and is a resource to help interpret the PBMAS district reports. It can be downloaded from: www.tea.state.tx.us/pbm.

4. PBMAS 2004-2005 Special Analysis • A NEW process for the monitoring system • Based on one of the system’s guiding principles, MAXIMUM INCLUSION : • PBMAS is designed to evaluate a maximum number of school districts through the use of appropriate alternatives for analyzing districts with small numbers of students.

5. PBMAS Special Analysis (cont.) • Special analysis does not apply to indicators for which the district has NO students in a particular group; these districts received a designation of No Data (ND). • Special analysis does not apply to indicators for which the district performance level is 0 – Met Standard, regardless of the student group size. • There are two types of special analysis in 2004-2005 PBMAS: automated special analysis and non-automated special analysis.

6. PBMAS Special Analysis (cont.) • Automated special analysis: • Performance level appears as 0SA, 1SA, 2SA, or 3SA on the PBMAS Report. • Applied to districts that did not meet the minimum size requirement of 30 in the current year, but did meet it over two years. (e.g., 14 students in the current year plus 16 students in the previous year). • The district received a performance level based on either the current year’s data or the previous year’s data, whichever resulted in the higher performance level.

7. PBMAS Special Analysis (cont.) • Non-automated special analysis: • Performance level appears as SA on the PBMAS Report. • Applied to districts that did not meet the minimum size requirement of 30 even when looking at two years. • The district will receive a performance level based on professional judgment. Summary data for the district are analyzed by program-area staff at the agency, and professional judgment is applied.

8. PBMAS Special Analysis (cont.) • The possible results of non-automated special analysis are: • Allowing the current year’s performance level based on small numbers to stand; • Elevating the current year’s performance level to a higher performance level; or • Determining that the district’s current year performance on the indicator should be Not Evaluated (NE).

9. PBMAS Special Analysis (cont.) • Update on non-automated special analysis • This year’s effort began during the middle of December. • Non-automated special analysis for CTE and NCLB have been completed. • Non-automated special analysis for BE/ESL and SPED are scheduled for completion at the end of January. • Districts that underwent non-automated special analysis will receive a summary report in February.

10. PBMAS Special Analysis (cont.) • This year’s timeline does not allow for non-automated special analysis on every PBMAS indicator. • The special analysis decision on certain indicators for this year is to assign a performance level of Not Evaluated (NE).

11. PBMAS Special Analysis (cont.) • Indicators that will receive a performance level of NE for 2004-2005 as a result of non-automated special analysis: • Across program areas (BE/ESL, CTE, NCLB, and SPED), all TAKS indicators for writing, socialstudies, and science. • In CTE: • CTE LEP TAKS Passing Rate (CTE #3) • CTE SPED TAKS Passing Rate (CTE #5)

12. PBMAS Special Analysis (cont.) • In BE/ESL: • ESL English TAKS Passing Rate (#1C) • LEP TAKS/SDAA Participation (#5) • LEP Year-After Exit English TAKS Passing Rate (#4A) • BE Year-After Exit English TAKS Passing Rate (#4B) • ESL Year-After Exit English TAKS Passing Rate (#4C) • In SPED: • TAKS Only Participation (#3) • SDAA Gap Closure (#7) • Year-After Exit TAKS Passing Rate (#14)

13. Review of 2004-2005 PBMAS Standards • Three types of indicators in the system: • Indicators with PLs based on an absolute standard • Indicators with PLs based on a relative standard • Indicators that are “report only.” • Absolute standards are preferred but not always possible. • When an absolute standard is not available, relative and “report only” are used.

14. Review of Standards (cont.) • Over time, relative standards will be replaced with absolute standards. • Relative standards are typically based on the distribution of scores of the population being evaluated. • The median serves as a guide in setting relative standards.

15. Review of Standards (cont.) • 2004-2005 PBMAS performance levels provide LEAs with an opportunity to review their own performance in relation to a standard, but performance levels are not risk levels and should not be interpreted as such. • Performance levels can be used by LEAs to begin to plan for strategies that will, over time, result in the LEA reaching the indicator’s standard.

16. Review of Standards (cont.) • PBMAS system, performance levels, and standards are new to ESCs and to districts. • Intervention framework acknowledges that the system is new and in transition. • The 2005-2006 PBMAS Manual will include a phase-in plan for standards. • Review of current standards and input from external stakeholders will be critical to the phase-in plan. • Phase-in of standards will enable ESCs and districts to better plan for improvement over time.

17. Review of Standards (cont.)