Researching Processes and System-level Change

1. Researching Processes and System-level Change Penny Hawe Markin Institute, University of Calgary phawe@ucalgary.ca

2. Take home message • When it comes to school-based health promotion, what’s inside the “black box” is still unclear/contested • Useless programs tell us important things • We need to shift from program thinking to system thinking 2

3. 3

4. 4

5. The Gatehouse Project: changes in health risk behaviour in year 8 students after 2 years All analyses adjusted for previous level of substance use in the school 5

6. Improved learning outcomes Emotional well-being INDIVIDUAL CHARACTERISTICS Sense of Belonging Connectedness Theoretical Framework of the Gatehouse Project Security Communication Positive regard Opportunities Skills Opportunities SOCIAL AND LEARNING ENVIRONMENTS 6

7. Social change processes matter. Social processes are easy to miss and elusive to track. But if we want to be more intentional and effective in creating supportive school environments, we’d better get better at measuring and recognising them. 7

8. TODAY • Looking inside the “black box” to understand change processes • Insights from “useless” programs • The promise of system thinking 8

9. Prevention Studies, Implementation FindingsBattish et al (1996) 66% of schools did not implement the program properlyRohrbach et al (1993) 79% teachers omitted program componentsTaggart et al (1990) 65% teachers did not implement properlyFlannery et al(1993) 67% teachers miss key components 9 Dulak JA, J Prev & Intervention in the Community 1998;17:5-18

10. Implementing preventive interventions Dosage Providing sufficient exposure to the program Adherence Following methods as outlined Quality of process Active methods to engage students Adaptation Modification to developmental or cultural needs + teacher attitude and experience Dusenbury et al Health Ed Res 2004;20(3):308-313 10

11. Prevention Studies, Implementation FindingsBattish et al (1996) 66% of schools did not implement the program properlyRohrbach et al (1993) 79% teachers omitted program componentsTaggart et al (1990) 65% teachers did not implement properlyFlannery et al(1993) 67% teachers miss key components 11 Dulak JA, J Prev & Intervention in the Community 1998;17:5-18

12. 12

13. Thinking of my teachers this term, I really like…….. All of them 14% Most of them 42% Half of them 16% One or two 25% None of them 3% 13

14. What is formally “taught” or implemented 14

15. Three perspectives on innovation Technological perspective Political perspective Cultural perspective House ER. Three perspectives on innovation. In Lehming, R., & Kane, M. Eds Improving Schools. Using What We Know. : Sage 1981 15

16. School Staff and Teacher Network Survey Assessed - knowing by name - regular conversations - knowing more personally - advice seeking - socialising Twice. At the start, and one year later. 16

17. From the list below, tick of the name of the people you talk with regularly (more than just saying ‘hi’) Ahmed Abboud Sue Smith Kyle Hilllary Gideon Gluckstern Nick Quinn Tim Blythe Salim Noosar Yvette Lemieux Josee Giscard Mitzi Lamarack 17

18. 18

19. Advice seeking a year ago 19

20. Advice seeking now 20

21. Socialising with each other a year ago 21

22. Socialising with each other now 22

23. Density of relationships, before and after (%) Before After Knowing by name 66 95 Knowing more personally 29 39 Regular conversations 26 41 Seeking advice 15 21 Socialising with 6 8 23

24. Two step reach for key actors 24

25. A good result But it always depends on what type of networks are desirable: - dense versus sparse - reciprocity - centrality of particular people ..…it depends on your theory and objectives 25

26. TODAY • Looking inside the “black box” to understand change processes • Insights from “useless” programs • The promise of system thinking 26

27. DARE Drug Abuse Resistance Education • 33 million children 1983-1997, no evidence of effectiveness • Costly. Average of $217-$334 per child per year • Renovated at cost of $13.7m in 2001, still not known if effective Example 1 27

28. “One of the most important benefits and by products is the relationship with the school department…. It really couldn’t be better. If ever I need anything all I have to do pick up the phone.” Police Chief Evaluation and Program Planning 2002;26:575-603 28

29. Example 2 29

30. Physician innovator whose actions people want to reward, praise • Manager wants to inform nurses of how some new thing should get done, so a leaflet with ‘dual’ purpose gets designed • Committee formed to develop a leaflet develops a solution that reflects everyone’s interests • Public relations department updates, and makes more attractive, a leaflet designed by rival agency 30

31. TODAY • Looking inside the “black box” to understand change processes • Insights from “useless” programs • The promise of system thinking 31

32. You know you have changed a person when………. You know you have changed a system when………….. 32

33. System thinking…. Inter-connections, interactions Local contextual meaning Feedback loops Amplification mechanisms Status of the parts of the system engaged in the intervention 33

34. Essence of ecological system thinking Properties of social contexts keep replicating particular health outcomes...regardless of who is cycled through these contexts …..social roles, work roles, classrooms, schools…. 34

35. Research implications