A12: Roundtable/Networking Evaluation Tools

1. A12: Roundtable/Networking Evaluation Tools Presented by Peg Mazeika& Kao Moua Her WI PBIS & RtITechnical Assistance Coordinators Amy Tranel External Coach MMSD

2. Today’s Agenda • Overview of PBIS and Academic Surveys and Tools on the PBIS Apps website and Wisconsinrticenter.org website • Evaluation Action Plan • Generate discussions, decisions, and selections on best fit tools • Collaborate and develop district and/or school timelines • Yearly Action Plan • 3-5 Year Plan • Evaluation Schedule Examples • Questions/Discussion

3. Why Evaluate? Synthesize Upon • What is the current status of our implementation? • To what extent was Academic RtI/SWPBS implemented as designed? • What do we need to attain/maintain fidelity? Move up to next tier? What Training, Coaching, Staff Training? Family/Community support? What funding will be needed? • Address the evaluation question: “Are we doing what we said we would do?”

4. Stages of Implementation Implementation Takes Time: 2 – 4 Years Exploration Installation Initial Implementation Full Implementation/ Continuous Improvement

5. Levels of Implementation *Exploration *Installation Not in Place Sustainability

6. DistrictLeadership TeamSelf-Assessment • Use by district leadership team (for example, district administrators, principals, external coaches, etc) • Use to self-assess RtI Academic & Behavior overall for annual and long term goals and needs • Develop action plan for training, funding, Coaching, professional development, technical assistance for team and staff

7. What is Evaluation? “Evaluation is the process of collecting and using information for decision-making. A hallmark of School-wide Positive Behavior Support (SWPBS) is a commitment to formal evaluation.” District Leadership Team Self Assessment Wisconsinpbisnetwork.org Appendix A Blueprint For Success

8. Current Tools for School PBIS Implementation Evaluation

9. Current Tools for Academic RtI Implementation Evaluation School-wide Implementation Review & RtI All-Staff Perception Survey

10. Current Tools for Academic RtI Implementation Evaluation School-wide Implementation Review & RtI All-Staff Perception Survey

11. Current Tools for Academic Implementation Evaluation: School-wide Implementation Review

12. Current Tools for Academic Implementation Evaluation: RtIAll-Staff Perception Survey

13. www.pbisapps.org wisconsinrticenter.org

14. Decisions….Decisions! • Decisions about Evaluation Tools • Which tools answer your district/schools’ evaluation questions/needs? • How will you use data to action plan district/team training and support needsbased on evaluation results? • Decisions about EvaluationTimeline • When/how often do you administer the tools? • Decisions about Evaluation Reporting • Who & How Often will you report outcomes to school staff and to the District Leadership?

15. Evaluation & Action Planning Decisions about Evaluation Tools • Does your District/School need a Research tool? • PBIS: (SET/ISSET) • Academic: School-wide Implementation Review (Reading/Math) & RtI All-Staff Perception Survey (Reading/Math) • Which Annual Assessment Tool(s) will you use? • PBIS: BoQ, ECBoQ SAS, Safety Survey, BAT, • Academic: School-wide Implementation Review (Reading/Math) & RtI All-Staff Perception Survey (Reading/Math) • Which Screening/Progressing Monitoring Tool(s) will/do you use? • PBIS: TIC, MATT • Academic: School-wide Implementation Review (Reading/Math) & RtI All-Staff Perception Survey (Reading/Math) • Does your District/School plan include evaluation of all levels of implementation? Tier I (Universal Level), Tier II (Selected Level) -Tier III (Intensive Level)?

16. Example:The District wants to bring RtI Behavior and Academics together and create one evaluation plan. • In what specific PBIS & Academic RtI areas are we strong, and in which areas do we need improvement or training? • Does our current evaluation plan meet the needs of our school(s) in both Academic and Behavior? • How can we schedule our evaluations so that we are not overwhelming our schools? • What Funding/Coaching /Training /Technical needs will we have in the next few years to scale up our implementation in both areas?

17. Use your ABCs: Existing Data

18. Behavior & Academic Assessment Timeline Adapted from Nieves Flores, 2012

19. ExampleDistrict/School Evaluation QuestionsNewly implementing/not yet at fidelity; Questions to be answered: • What is our baseline, and what is our progress through the year in our Systems development and implementation? • Are we improving on our implementation compared to baseline? Moving toward attaining Fidelity? Are student outcome data improving? • How do our individual staff members rate our implementation in each area? • Are our self-assessments consistent with data from outside evaluators? (SET, Walk Through) • Are we meeting our annual Action Plan Goals?

20. ExampleDistrict/School Evaluation QuestionsImplementing PBIS for three years: We have been using the SAS, BoQ, SIR and have started Tier 2. • What specific areas in RtI are we strong in, and in which areas do we need training or support to improve? • Are we maintaining our implementation of RtI Academic/PBIS at Tier 1 as we move up in the Tiers ? • Do our staff members continue to ‘buy in’ to our Academic/PBIS implementation? • Have we assessed our implementation of in the areas of Culturally Responsive Practices and Family Engagement?

21. Task: Developing your Plan Questions to Consider: • Which surveys are your schools/districts currently using? • Do they answer your evaluation questions: • Are we implementing RtI/SWPBIS with fidelity? (SAS, BoQ, BAT, SIR) • Is there more information that we need? (TIC,MATT, SSS, CRP Interview, Family Engagement Interview) • How does our staff rate our implementation (SAS, RtI All-Staff Perception Survey) • Do your surveys assess ALL the Tiers you are implementing? (TIC, ECBoQ, BoQ, MATT, BATT, TFI) • Do you need a research tool? (SET, ISSET) • Are you using a Universal Screener for Behavior/Social Skills? • Are you Action Planning from your Assessments?

22. Task: Developing your PlanOther questions to consider • Are you “topping out” of the surveys you are currently taking? • Do you need to customize your evaluation plan for your school, or will one evaluation plan fit for all schools in your district? • Are you assessing both Academic and Behavior RtI implementation? • Do you use Universal Screening? • Is your Evaluation Plan manageable for your school(s)? • Do you have a District Leadership Team that plans for PBIS/RtI Implementation and support? • Does your District Leadership Team Self Assess annually?

23. Task: Complete this Plan for your District/School

24. Action Plan from your Evaluations! Action Plan Resources: Templates on PBISassessment.org for TIC, SAS & BoQ etc. action plans Use Action Plans available in resources area Wisconsinpbisnetwork.org TIPS format meeting minutes action plan State specifically when you will review your goals in your action plan!

25. Identify components you are Working on Now! What RtI components are already underway?

26. School name_________________________________ Date______________Implementation Action Plan:

27. Universal Screeners For info on Selection of Universal Screeners Go to Wisconsinpbisnetwork.org enter Universal Screeners in Search box

28. Resources: • pbisapps.org: website to connect to pbis assessment materials , videos, resources • Evaluation Blueprint for School-Wide Positive Behavior Interventions & Support: Appendix A District Leadership Assessment Tool (wisconsinpbisnetwork.org) • Bob Algozzine, Robert H. Horner, George Sugai, et.al The Role of Fidelity Measures in Implementing Evidence-based Practices: Lessons Learned from School-wide PBIS • Tary J. Tobin , Claudia G. Vincent, et.al