DE-PBS Key Features Evaluation: Matching Philosophy & Measurement

1. DE-PBS Key Features Evaluation: Matching Philosophy & Measurement Sarah K. Hearn, M.Ed., Delaware Positive Behavior Support Project OPTIONALLOGO HERE Development Goals Item Breakdown Rubric Samples Levels by Evaluation Section N=22 • School-wide PBS Tier 1: Program Development & Evaluation • Office Discipline Referral data pulled by a designated person, reviewed monthly by the School-wide team, and shared with entire staff. • Source: Team Leader Interview - Does the PBS team have a designated data person who is able to pull/print reports for monthly meetings (Big 5)? Are ODR data collected and reviewed in summary form monthly by the PBS team? How often are the Big 5 data summaries shared with all staff during the school year? How are they shared? • Prevention: Implementing Schoolwide & Classroom Systems • Students are recognized for their good behavior (e.g., verbal praise, coupon, privilege), AND students can state the reason for the recognition. • Source: Student Interview - Have you received any recognition from a teacher/adult in school for good behavior during the past week? If yes, why did you receive this recognition? • Correcting Problem Behavior • Correction is viewed not just as use of punishment or consequences but also as opportunity to help develop social problem-solving/decision-making skills of self-discipline and prosocial behavior. • Source: Staff Interview - Beyond classroom management strategies, when correcting misbehavior, how do you help develop problem-solving & decision-making skills? • Developing Self Discipline • Social Emotional Lessons are infused throughout the school curriculum. • Source: Staff Interview - Do teachers or other staff teach lessons intended to help develop social/emotional competencies? • Fully align external evaluation with DE-PBS Key Features • Include qualitative & quantitative data • Link to Professional Development • Include multiple key perspectives in process • Differentiate implementation levels of SWPBS to better describe schools and provide appropriate supports Development Process • Rubric development to map out components of each of the 10 Key Features of School-wide Positive Behavior Support in Delaware • Inclusion & expansion of some School-wide Evaluation Tool (SET) components • Determine information sources • Question development • Peer review & rubric reorganization to match existing tools • Spring 2010 – Question pilot • Spring 2011—2013 – School-based pilot phases & revisions • Phase 1 – 2 schools • Phase 2 – 7 schools • Phase 3 – 28 schools Scoring & Reporting • Information from multiple sources used to score all rubric components • Scale = 3, 2, 1, 0 • Per rubric, a set of items identified as “Essential Items” – Defined as key system elements & illustrate evidence of implementation fidelity • Scoring based on Essential Items • Reporting provided on all rubric areas • Narrative summary organized by the 4 main evaluation areas which include: • Areas of strengths • Recommendations for improvement, professional development opportunities and resources • Report review tool & action planning guide Revisions & Findings to Date • Revisions: • Modify questions based on consistent scoring concerns • Condense and streamline rubric areas • Revisit rubric definitions, especially on essential items, to ensure clarity and support consistency • Explore level parameters due to large percentage of Developing scores • Findings: • More schools reaching Proficient and Exemplary levels in Program Development • 50% of schools at Exploring in Correcting Problem Behaviors = Identified need for additional professional development Evaluation Procedures • On-site visit by DE-PBS Staff • Sources of Information: • Interviews with administrator, SW team leader, teachers/staff, students • Review of SW PBS process documents • Schoolwide observations • Data: School Climate Surveys, DE Assessment of Strengths & Needs for PBS • Use of iPads or Paper/Pencil SW-PBS Implementation Levels – Under Refinement Next Steps • Tool Revisions: • Analyze data set with all Spring 2013 data • Finalize level system & notify schools • Finalize all items and questions • Testing for tool validity and reliability • Finalize Filemaker application for iPad • DE-PBS Project Plans • Support schools to better implement the programs developed • Focus on Correcting Behavior Problems in training and technical assistance – at district coach and school levels • Explore implications for categorizing schools using new level system Structure Contact Information Sarah K. Hearn – skhearn@udel.edu www.delawarepbs.org