Benchmarks of Quality BoQ

1. Benchmarks of Quality BoQ Lisa Leonard, PBIS Facilitator lrleonard@ssdmo.org

2. Entering Activity Our team measures fidelity of implementation of universal systems of PBIS in the following ways: Place a dot what your team does to measure fidelityPlace a dot what your team does to measure fidelity

3. Agenda BoQ Overview BoQ Items and self-assessment Using the BoQ for universal action planning Walk -through with BoQ Social Validity Survey In packet Rating Form, Scoring Guide, Action Plan, include Walkthrough in presentation, In packet Rating Form, Scoring Guide, Action Plan, include Walkthrough in presentation,

4. Purpose The Benchmarks of Quality is a self-assessment process in which the PBIS internal coach will evaluate the development and implementation of the critical features of School-Wide PBIS. The BoQ measures fidelity of implementation, within 10 critical elements of SW-PBIS The BoQ is designed to give your school feedback for future action planning and is used to determine what level of professional learning in PBIS will be recommended to your school team

5. Objective: What you can expect to learn To examine the fidelity of implementation of the universal level Are we really doing what we think we are doing? To document the effectiveness of implementation Is what we�re doing working? To identify and examine strengths and weaknesses of implementation Celebrate successes, identify areas to improve

6. Where You Have Been Formative Assessments Team Implementation Checklist (TIC) Self-Assessment Survey (SAS) School Safety Survey (SSS) Summative Assessments School Wide Evaluation Tool (SET) Benchmarks of Quality (BoQ) Benchmarks of Advanced Tiers (BAT) Individual Student Systems Evaluation Tool (ISSET)

7. Why the BoQ and not the SET? Met the 80% criteria on the SET for two years Self-assessment heightens awareness Takes less time SET may have become routine SET scores may be inflated and may not reflect current practices

8. Benchmarks of Quality Developed by Florida PBIS at the University of South Florida. We have adapted the scoring to best meet schools needs. Assesses development and implementation of school-wide PBIS Lists 53 benchmarks of quality in school-wide PBIS programs within 10 critical elements Completed annually by school teams to identify areas of strength and weakness

9. Completion of BoQ Completed by the Universal SW-PBIS Team Reflects the consensus/majority of team members present (universal coach, administrator, Tier 2/3 systems coach) For items completed on which there is disagreement, team discusses and comes to score Should be completed by team members individually; facilitated by the internal PBIS coach; results compiled by internal PBIS coach Summative assessment, may want all to complete and compile results or you may want to implement like a TIC and look for agreement Summative assessment, may want all to complete and compile results or you may want to implement like a TIC and look for agreement

10. Sections of the BoQ 53 items aligned with PBIS training process addressing areas of: PBIS Team Faculty commitment Effective procedures for dealing with discipline Data entry and analysis plan Expectations and lesson plans developed School-wide recognition system established Lesson Plans for teaching expectations Implementation plan Classroom Systems Evaluation

11. Completing the BoQ More specific than the TIC use the Scoring Guide as a learning tool BoQ results are used for action planning. Use of sub-scores to focus and refine action planning Try to stay true to your team�s functioning, this is a summative measure to guide your universal team�s work

12. Scoring Items Using the rubric, the team will match their universal implementation to the scoring guide for the BoQ Scoring guide is based on 3,2,1,0 point scale, not all questions are worth the full value range. Sub scores will be calculated after each of the ten sections. A total score will be calculated at the end based on a total of all sub scores

13. Scoring the Sections At the end of each section is a sub section scoring box We will be going through each section to explain difficult questions and field questions on items. After each sub section we will be calculating a sub score. This will be based on the points earned from each section. Points earned will differ from question to question.

14. Assessing using the BOQ Question Explanations

15. PBIS TEAM 2. Team has regular meetings (at least monthly) Two Points- Team meets monthly (min. of 9 one-hour meetings) This means August to April or September to May Summer planning prior to school may be eligibleSummer planning prior to school may be eligible

16. Faculty Commitment 5. Faculty involved in establishing and reviewing goals Have you shared your SET, Self-Assessment, School Safety results with the rest of the faculty? 6. Faculty feedback is obtained throughout the year Given opportunity to provide feedback for every step of the PBIS process

17. Effective Procedures for Dealing with Discipline 8. Discipline process includes documentation procedures Are minor and major misbehaviors differentiated and tracked? 12. Suggested array of appropriate responses to major (office managed) problem behaviors All administrators use the menu of selected evidence-based appropriate responses

18. Data Entry and Analysis Plan Established

19. Expectations and Rules Developed

20. SW Expectarions, rules for setting and all settingsSW Expectarions, rules for setting and all settings

22. Expectations and Rules Developed 21. Staff are involved in development of expectations and rules Knowing that all staff have been involved or been offered the opportunity for equal voice. Staff have guided work, revisions to work

23. Reward/Recognition Program Established 23. A variety of methods are used students Free and frequent Intermittent Long term 26. Ratios of acknowledgement to corrections are high Examples of how you might capture the ratio (ticket/data) collection

24. Reward/Recognition Program Established 28. The system includes incentives for staff/faculty Continuum of incentives, not always administration based, examples

25. Lesson Plans for Teaching Expectations and Rules 32. Lessons are embedded into subject area curriculum Alternating classes to not interrupt instructional time, Academic lab time or advisory, weekly or daily. Using common language, feedback, quick error corrections and re-teaching of expectations

26. Implementation Plan 37. A plan for teaching students expectations/rules/rewards are developed and scheduled and delivered Plan is developed for throughout the year, not the first two weeks of school. Team anticipates changes to the incentive system. Teaching plan is data driven. 38. Booster sessions for students and staff are planned, scheduled and delivered

27. Classroom Systems 46. Acknowledgement of students demonstrating adherence to classroom rules and routines occurs more frequently that acknowledgment of inappropriate behaviors Teachers need to make positive academic and behavioral statements at a rate of 4 positive comments to every 1 corrective statement

28. Classroom Systems 47. Procedures exist for tracking classroom behavior problems Classroom behavioral data is kept

29. Evaluation

30. Scoring the BoQ Add up your total score for the end of each of the ten sections. Place your total score, (the sum of your sub scores) in the total box at the bottom of the page.

31. Range of Scores Florida�s PBIS states that: BoQ score of 70% or greater represents implementation is consistent and ongoing across staff as they teach, reinforce and engage in proactive discipline practices BoQ score of 70% or lower displays a need to continually revisited universal implementation and classroom management procedures After reviewing you School-Wide Big 5 Data if you have data that is increasing, and you received a 70% or higher on the BoQ, you need to stay at universals and examine your universal systems approach

32. Range of Scores for BoQ BoQ is based out of a total of 107 points. 80% is 86 total points Looking at your total score you also want to look at your Big 5 data, although you may have an 70% or higher on the BoQ if your incidents of inappropriate behavior are increasing, you want to examine your universal systems.

33. Action Planning Looking at your sub scores select the 2-3 sections you scored the lowest on or feel need improvement. These will be your action planning topics. If the topic did not apply to your school, select the point your team feels needs to be worked on. Example: Early Childhood may scored low in Data Entry and Analysis Plan due to lack of an Office Discipline Referral form. May select another sub section.

34. Action Planning Using your universal 2010-2011 Action Planning Sheet (in your packet) add to your action plan with skill steps on how you will achieve the action you selected.

35. School-Wide Action Plan2010-2011

36. Benchmarks of Quality Walkthrough Procedures

37. Purpose This walkthrough is designed to give school teams a glance of school-wide PBIS. This tool will allow the observer to provide feedback to the SW-PBIS team and administration on implementation. This walkthrough will be completed by a SSD PBIS Facilitator and is a required for the Missouri PBIS Recognition Program.

38. Interview Section The administrator will be interviewed. This interview will take 7-10 minutes and there are 8 interview questions.

39. Products to be Collected The Big 5 Data report will be collected. We are looking for the report that is less than one month old. If you are in February, you would like to see February or January�s report.

40. Walkthrough Five school-wide sites will be observed (hallway, playground/gym, cafeteria, restroom, other) looking for postings. Five classrooms will be observed.

41. Interviews Five students will be interviewed. A random sample including one special education student should be included. Five staff members will be interviewed and a six questions will be asked of staff. This should take between 3-5 minutes per staff member.

42. Social Validity Survey Please complete the Social Validity Survey. This will help our team focus its efforts and guide our work.

43. Thank You! Thank you for your time and dedication to SW-PBIS! As part of your evaluation process, your internal PBIS coach and principal will be receiving an email outlining your evaluation procedures, this will have the individualized assessment descriptions, social validity survey link, and contact information from the SSD PBIS facilitator team.