Working with Data

1. Working with Data Mankato Coaches’ Meeting December 1, 2011 Marshall Coaches’ Meeting December 6, 2011

3. “Never ask for data that are not being used to make decisions.” Rob Horner, Co-Director National Center on SW-PBIS 2011

4. Overview • Why use data? • What are the classes of data? • What are data-based procedures?

5. Using Data • Data take the emotion out of our experiences • Data let us know if our system and practices are effective • Data remind us to celebrate our successes • Graphs help people see the big picture • Concept & logic of problem solving • Defining the problem • Setting goals • Using data is fun!

6. “You have to do something about these tardies!!!!”“Everybody is tardy!!!”

7. Graphs help people see the big picture • It takes the emotion out of our experiences

8. PBIS in a Nutshell: Where Do Data Fit in? Outcomes Social Competence & Academic Achievement Data Practices Systems Supporting Student Behavior Supporting Staff Behavior Supporting Decision Making

9. Two simple questions • Did we do what we said we would do? • Did it make a difference?

10. In the beginning… There were….. stages of SW-PBIS implementation And measures of…. effort, fidelity, outcomes You have…. TIC, SAS, ODR, and SET…

11. Stages of Implementation Exploration/Adoption Installation Initial Implementation Development Commitment Elaboration Establish Leadership Teams, Set Up Data Systems Continuous Regeneration Provide Significant Support to Implementers Embedding within Standard Practice Improvements: Increase Efficiency and Effectiveness Where are you?

12. Classes of Data • Effort data (Are we working the plan?) • Team Implementation Checklist (TIC) • Self-Assessment Survey (SAS) • Outcome data (Is it having an effect?) • “Big 5” Graphs • Triangle % • Fidelity data (Are we following the plan?) • School Evaluation Tool (SET) • Benchmarks of Quality (BoQ)

13. Now You have…. TIC, SAS, SET, and ODR..

14. In the beginning • Establish evaluation infrastructure • (data tools that are readily accessible) • Establish routines • Learning how to ask the questions • How, when and what data to gather • Practice

15. Logic of problem solvingwithin decision-making systems

16. Improving Decision-Making From Problem Solution To Action Planning Solution Problem Problem Solving

17. Main Ideas • Decisions are more likely to be effective and efficient when they are based on data. • The quality of decision-making depends most on the first step (defining the problem to be solved) • Define problems with precision and clarity

18. Main Ideas • Data help us ask the right questions…they do not provide the answers: Use data to • Identify problems • Refine problems • Define the questions that lead to solutions • Data help place the “problem” in the context rather than in the students.

19. Main Ideas • Build “decision systems” not “data systems” • Use data in “decision layers” • Is there a problem? (overall rate of ODR) • Localize the problem • (location, problem behavior, students, time of day) • Get specific • Don’t drown in the data • It’s “OK” to be doing well • Be efficient

20. Defining the problem

21. Using Data to Refine Problem Statement • The statement of a problem is important for team-based problem solving. • Everyone must be working on the same problem with the same assumptions. • Problems often are framed in a “Primary” form, that creates concern, but that is not useful for problem-solving. • Frame primary problems based on initial review of data • Use more detailed review of data to build “Solvable Problem Statements.”

22. Precise Problem Statements(What are the data we need for a decision?) • Precise problem statements include information about the Big Five questions: • What is problem, and how often is it happening • Where is it happening • Who is engaged in the behavior • When the problem is most likely • Why the problem is sustaining

23. Examples: Primary to Precise • Gang-like behavior is increasing • Texting during school is becoming more negative • Bullying (verbal and physical aggression) on the playground is increasing during “first recess,” is being done mostly by four 4th grade boys, and seems to be maintained by social praise from the bystander peer group. • A large number of students in each grade level (6, 7, 8) are using texting to spread rumors, and harass peers. Texting occurs both during the school day, and after school, and appears to be maintained by attention from others.

24. Primary Statements Too many referrals September has more suspensions than last year Gang behavior is increasing The cafeteria is out of control Student disrespect is out of control Precision Statements There are more ODRs for aggression on the playground than last year. These are most likely to occur during first recess, with a large number of students, and the aggression is related to getting access to the new playground equipment. Primary versus Precision Statements

25. Primary Statements Too many referrals September has more suspensions than last year Gang behavior is increasing The cafeteria is out of control Student disrespect is out of control Precision Statements There are more ODRs for aggressionon the playground than last year. These are most likely to occur during first recess, with a large number of students, and the aggression is related to getting access to the new playground equipment. Primary versus Precision Statements

26. Precise or Primary Statement? • Children are using inappropriate language with a high frequency in the presence of both adults and other children. This is creating a sense of disrespect and incivility in the school. • ODRs during December are higher than in any other month. Primary Primary

27. Precise or Primary Statement? Our school isn’t following SW-PBIS, although we have some good practices in place. Our school has some features of SW-PBIS in place including administrator support, more than 80% of total staff support, and an established team. We need to review these commitments, establish regular meeting times, review our current TIC and SAS, conduct a resource review and begin to identify our 3-5 expectations.

28. Precise or Primary Statement? • James D. is hitting others in the cafeteria during lunch, and his hitting is maintained by peer attention. • Boys are engaging in sexual harassment. • Three 5th grade boys are name calling and touching girls inappropriately during recess in an apparent attempt to obtain attention. Precise Primary Precise

29. Precise or Primary Statement? Minor disrespect and disruption are increasing over time, and are most likely during the last 15 minutes of our block periods when students are engaged in independent seat work. This pattern is most common in 7th and 8th grades, involves many students, and appears to be maintained by escape from work (but may also be maintained by peer attention… we are not sure). Precise

30. So….what… Are the questions?

31. Roadmap

32. Are we implementing SWPBIS?How will we know? • TIC • SAS

33. TIC 3.1 (effort)

34. Create a picture & story-TIC • Use data to create a story and give a picture

35. Item Scores-TIC

36. Story • Is this school implementing SWPBIS? • What progress did this school make with implementation of SWPBIS • What goals/tasks might be in their action plan? • How would you coach the team? • How would you share this story with the whole school?

37. Self-Assessment Survey (Effort)

38. Individual Summaries Self-assessment system • School-wide • Non-classroom • Classroom • Individual

39. Individual Summaries

40. Individual Summaries

41. Analysis of School-Wide System

42. Individual Item Scores

43. Story • Where is this school with SWPBIS implementation? • How do staff perceptions compare to the PBIS Leadership team? • How would you coach the team? • What things might be in their action plan? • How would you share this with whole school?

44. Are we implementing with fidelity?

45. Fidelity Data:SET Results Your primary target is 80% Average Your secondary target is 80/80 80% teaching 80% average

46. Does it make a difference?

47. Identifying problems/issues using ODR data • What data to monitor • ODR per day per month • OSS, ISS, Attendance, Teacher report • Team Checklist/ SET (are we doing what we planned to do?) • What question to answer • Do we have a problem? • What questions to ask of Level, Trend, Peaks • How do our data compare with last year? • How do our data compare with national/regional norms? • How do our data compare with our preferred/expected status? • If a problem is identified, then ask • What are the data we need to make a good decision?

49. The “Big 5” • Most questions can be answered using the “big 5” reports • The “big 5” help you define the problem for your context. • Average referrals per day per month • Problem behaviors • Location • Student • Time

50. Problem The Big 5 ODR Per (day, student, location, time, behavior)