The Effect of External Mandates On Internal Capacity

1. The Effect of External Mandates On Internal Capacity David Bower, Ed.D. Ohio University Athens, Ohio November 2005

2. Background 2001-2003: doctoral research on complexity theory, self-organization, and leadership 2003: Leadership and the Self-Organizing School, a qualitative study on self-organization in a middle school 2003-2005: new questions about effects of high-stakes testing, accountability, school ratings, and other mandates and their effect on internal capacity to self-organize

4. Pilot Study: Fall, 2005 Selection of two middle schools � Central Middle School and Southeast Middle School 10 interviews with teachers (5 per school) All names changed Goal: to understand issues related to effects of mandates, and especially effects of high-stakes testing

5. Open-ended interview topics Where is the source of inspiration, or a catalyst, for change? Does the school look within or without for ideas? Do staff members internalize the goals of the school? If so, how? How do staff members make sense of school activities, goals, and improvement initiatives? Do staff members learn about the environment outside of the school? If so, how, and how is this knowledge used within the school? Are school initiatives sustained or episodic?

6. Ohio School Rating System � Excellent schools meet 94% or more of applicable indicators or 100 or above on the Performance Index (PI). � Effective meet 75% to 93.9% of applicable indicators or score 90 to 99.9 on the PI. � Continuous Improvement meet 50% to 74.9% of applicable indicators or 80 to 89.9 on the PI OR they meet AYP (the lowest a district can be rated if they meet AYP is CI). � Academic Watch schools meet 31% to 49.9% of applicable� indicators or score 70 to 79.9 and have missed AYP. � Academic Emergency schools are those that met 30.9% or fewer indicators, scored less than a 70 and missed AYP.

7. School profile: Central Middle School Serves grades 7 and 8 About 440 students Called a middle school but functions as a junior high school Only middle level school in town Rated as �Effective� during 2004-2005 academic year District also rated as �Effective�

8. School Profile: Southeast Middle School Serves grades 6-8 About 500 students Called a middle school; functions more like a junior high school but with some middle school features (loosely organized teams) Only middle level school in rural district Rated as �Continuous Improvement� during 2004-2005 academic year District rated as �Continuous Improvement�

9. Limitations Limited scope of study (two schools, 10 faculty members) Administrators not available as of November for interviews Participants are newer teachers Veteran teachers (described as complacent by participants) did not volunteer to participate

10. Initial conclusions: Business as usual Lack of clearly articulated school-wide goals Episodic and fragmented response to high-stakes testing Responses and urgency linked to school ratings (high rating = high complacency) Leadership is a key factor �Continuous improvement� may mean �continuing to do the same thing� with no improvement Complacency

11. Where is the source of inspiration, or a catalyst, for change? Does the school look within or without for ideas? Lack of any coherent goals at either school Limited use of test data at Central MS Regular discussion of test data at Southeast MS Weak or non-existent links with elementary schools and high school within district Awareness of community profile and demographics but little connection to school goals

12. Hannah Acker, Central Middle School The only time we have looked at test data is when we adopted text books We�re not at academic watch or emergency. We�re on solid ground. I don�t feel like I�m under a microscope. And it�s kind of like, we read articles in the paper about schools around us that aren�t, and it�s a huge sigh of relief that we�re not under the microscope like they are, or not being published in the paper. ��it�s business as usual.

13. Do staff members internalize the goals of the school? If so, how? No goals to internalize Vague sense of personal place/purpose within the organization Personal conversations about mandates, testing, and accountability

14. Maria Robbins, Central Middle School And what does come through is �oh by the way�� It�s all word of mouth, all informal, nothing written. There�s no kind of communal culling of information or processing. If there�s no emergency, then no one acts in any urgent mode. We anticipate coasting, but no one would ever label it as coasting. Business as usual, perhaps. Because we�re doing fine.

15. How do staff members make sense of school activities, goals, and improvement initiatives? Information is informal (almost non-existent to some) at Central Information is informal and indirect (principal to grade representatives to teachers) at Southeast Sense-making is individual rather than collective

16. Valerie Lane, Central Middle School I wish this school was more single-minded. I think that�s important. We�re a team, we�re here to win, we want the students to win. It�s just like parenting. When you have both parents working together it doesn�t confuse the kids � the message is the same.

17. Do staff members learn about the environment outside of the school? If so, how, and how is this knowledge used within the school? Limited information from school administration and central office Learned about ratings, mandates, etc. from news media Limited discussion within school

18. Valerie Lane, Central Middle School People aren�t talking about the ratings � yet � but I know all the telltale signs because I was at other schools. Like, last year, for the first time during home room we were doing math problems that the math teachers had provided. And I was laughing a little, because I wanted to rebel, because this doesn�t help, and it�s too late, and you have a wide range of kids in home room, and I don�t teach math. This reminds me of a hamster on a wheel. The hamster feels like it has to go somewhere, so it runs faster. And when I am seeing all of these things done, it�s just a crazy cycle.

19. Are school initiatives sustained or episodic? Episodic or non-existent �More of the same� (more math homework to improve math skills) No cross-curricular initiatives No whole-school review of work

20. Anna Gibson, Southeast Middle School There would be a ripple and there would be some band-aids. I don�t know if it would be enough. It�s almost like you have to step back and see the big, big picture and try to grasp it. You can put band-aids here and there, and you can look at individual teachers or individual classes or individual buildings, and put those band-aids on, but if you don�t put them everywhere � like one big overhaul � it won�t help. And it would probably have to be someone from outside the district. But those folks don�t have the inner picture. They can say �do this, or do that� but if they don�t know what the climate is, those suggestions aren�t going to be valid either.

21. Conclusions from pilot study Initial question: What is the effect of mandates on internal capacity? New questions: What are mediating factors that affect response within schools? What information gets in and how is it filtered? How does this relate to the conceptual model?

23. Conclusions from pilot study Conceptual framework model: Role of boundaries - what information gets through and how is it filtered? Core values - how and when do these change?

24. Conclusions from pilot study Factors affecting response: School rating -higher rating may result in complacency; lower rating may result in urgency Teaching experience - new teachers may be more willing to respond Leadership-may determine what is seen as a priority and how to respond; buffering can result in �business as usual� response

25. Is this relevant? 2005 NAEP results chart

26. Next steps? Questions: What is role of institutional support? Is the response for improvement simply �Teach better?� What is next after NCLB? Next study?