Accountability & Evaluation

1. Accountability & Evaluation

2. Overview of this Session • Defining Accountability& Evaluation • MTSSS & Program Evaluation Issues • Example of Evaluation in MTSSS • Discussion • Current status • Next steps

3. What does… • Accountability mean to you? • Evaluation mean to you?

4. Accountability in Florida • Increasing accountability focus the last decade • Examples include: • School grading • AYP • Special education rules • DA • FEAPS & Teacher evaluation systems

5. Impact of Accountability Criticisms Positives Establishes and maintains standards for performance Reinforces use of data to monitor student outcomes Reinforces need to examine resource use Student outcome rather than process focus Success stories • Lack of educator involvement • Controversy • Consequence driven • Compliance driven • Conflicting requirements • Duck and cover approach

6. (Hall & Hord)

7. Accountability & Evaluation Issues • Compliance driven versus informative evaluation • Evaluation often done to meet accountability requirements • Evaluation can serve to help integrate and improve school and district services • Evaluation is fundamental to MTSSS • MTSSS has the potential to: • Be viewed as one more thing we have to do OR • Help address accountability & evaluation demands through the multi-tier framework

8. Mission and Vision Multi-Tiered System of Student Supports - Inter-Project Collaboration The collaborative vision of the Florida Problem-Solving/Response to Intervention (FL PS/RtI) and the Florida Positive Behavior Support/Response to Intervention for Behavior (FLPBS/RtI:B) Projects is to: • Enhance the capacityof all Florida school districts to successfully implement and sustain a multi-tiered systemof student supports with fidelityin every school; • Accelerate and maximize student academic and social-emotional outcomesthrough the application of data-based problem solving utilized by effective leadershipat all levels of the educational system; • Inform the development, implementation, and ongoing evaluation of an integrated, aligned, and sustainable system of service deliverythat prepares all students forpost-secondary education and/or successful employment within our global society.

9. Important MTSSS Evaluation Issues • Stakeholders should be involved in all aspects of planning and carrying out the evaluation process as well as in decision-making • Goals through planning should drive the process • Information obtained to: • Determine where you currently are (needs) • Take ongoing looks at how things are working • Make decisions about what to keep doing and what to change or eliminate

10. MTSSS Eval Issues cont. • The data you collect should be driven by the questions you want to answer • Are students meeting expectations? Academically? Behaviorally? Social-emotionally? • Are we implementing MTSSS with fidelity? • Do we have the capacity to implement successfully? • Do staff buy into implementing MTSSS? *Example questions

11. Example Questions and Data Sources

12. How Are Students Performing? Examples of data sources • Academics • FCAT • FAIR • Core K-12 • End of Course Exams • Behavior • Attendance • Tardies • Suspensions • Discipline referrals • Global Outcomes • Graduation Rates

14. www.flrtib.org

15. Are Schools Implementing MTSSS with Fidelity? Examples of data sources • Curriculum and Instruction/Intervention • Principal walkthroughs • Lesson plans • Intervention Documentation Worksheets • Components of MTSSS and Data-Based Problem-Solving* • BOQ, PIC, BAT • SAPSI, Tier I & II CCCs, Tier III CCCs * See http://flpbs.fmhi.usf.edu/ and http://floridarti.usf.edu for more information

18. Do We Have the Capacity to Implement MTSSS with Fidelity? Examples of data sources • Leadership Team structure and functioning • Organizational charts • Minutes/meeting summaries • SAPSI, BOQ, PIC • Staff knowledge and skills • FEAPs & teacher evaluation system • Staff development evaluations • Work samples • Resources allocated to match needs • SIP, DIP • Master calendar/schedule • School rosters • Resource maps

19. Marzano Evaluation Domains • Classroom Strategies and Behaviors • Preparing and Planning • Reflecting on Teaching • Collegiality and Professionalism

20. Do Staff Buy Into Implementing MTSSS? Examples of data sources • Leadership vision and commitment • SAPSI, BOQ, PIC • Required and non-required plans • Staff buy in • SAPSI, BOQ, PIC • District/school staff and climate surveys • Dialogue • Brief interviews with key personnel

22. Discussion • What are you currently doing to examine these areas in your district or school? • What are the critical questions you ask? • What data sources do you have to answer them? • What questions do you already have that you cannot answer with available data? • How do you use the data you collect to inform decisions? • What areas need to be addressed as you return to your districts to plan? What are the priorities? • What critical questions do you need to start asking? • What data sources do you need? • How can you better use the data to inform decisions?

23. Our Plan for Moving Forward • Learn more about your needs for evaluation support through needs assessment and DAPP protocol • Develop resources for evaluating critical domains necessary to implement MTSSS such as: • Leadership • Coaching • Fidelity of implementation • What additional supports can we provide?