School autonomy and student achievement. An international study with a focus on Italy

1. School autonomy and student achievement. An international study with a focus on Italy Angelo Paletta Maria Magdalena Isac Daniele Vidoni

2. The international evidence: • ICCS: • across countries, school autonomy does not have a significant effect on student civic knowledge; • however, the effect depends to some extent on the country context - although in most countries, the effects are nearly nil, in some contexts the parameters are significantly higher than the average effect . • PISA: • mixed results indicating both positive and negative effects of school autonomy on educational performance. • Differential effects might depend on the area of decision-making under investigation (what is taught, how student are assessed, publicity of results) and also on other characteristics of the educational system such as levels of development and accountability. • “autonomy affects student achievement negatively in developing and low-performing countries, but positively in developed and high-performing countries” (Hanushek E., Link S., Woessmann L. 2011)

3. The internationalevidence on the effect of school autonomy on civic knowledge

4. SchoolAutonomy in Italy: some remarks • BassaniniLaw 1997: schoolsacquirelegalpersonality • ConstitutionalReformof 2001: “The autonomousschools are the fundamentalbrickof the school system” (art. 117.m) • Autonomyis: • Curricular, • Organizational, • Financial (abilityofdiversifyingsourcesoffunding) • All teachers (~800K) are civilservantsemployedby the MinistryofEducation • To the point, schoolprincipalshipisnot a career per se. Rather, itis the “last stepof the teaching career”

5. School autonomy in ICCS and PISA: the dimensions and the space for action in Italy

6. Research questions • What is the relationship between school autonomy and multiple measures of student achievement (civic knowledge, math and reading) in the Italian context? • To what extent is the size and direction of this relationship generalizable across diverse outcome variables?

7. Method • Data sources: • The International Civic and Citizenship Education Study (ICCS) – 2009. • The ICCS data for Italy matched with national data on student achievement - INVALSI. • Samples: • International: 32 ICCS countries; 4423 schools; 116118 students. • Italian: 153 schools; 2599 students.

8. Data and Methods • Italian data - variables: • Dependent variables – Civic knowledge, mathematics and reading performance (cognitive tests - α > .80). • Explanatory variable - School autonomy • Control variables Student characteristics: age, gender, SES, language spoken at home School context: location, size, teacher student ratio, average SES. Method of analysis for Italian data: multivariate multilevel analysis (students within schools; regression model estimated for the three dependent variables simultaneously)

9. Results of multivariate multilevel analysis to explain variation in students Civic Knowledge, Italian language and Mathematics for Italy

10. Results of multivariate multilevel analysis to explain variation in students Civic Knowledge, Italian language and Mathematics for Italy • M1 • student control variables are related to student achievement in all domains. • Together, the control variables explain: • 17% of the total variance in civic knowledge, • 8% of the total variance in reading (Italian language) and • 2% of the total variance in the mathematics achievement

11. Results of multivariate multilevel analysis to explain variation in students Civic Knowledge, Italian language and Mathematics for Italy • M2 • school autonomy shows: • a statistically significant negative relationship (β = - 0.105, se = 0.033) with all three outcomes • the effect is common for all achievement measures and • explains nearly 1% of the total variance in civics, reading and mathematics

12. Results of multivariate multilevel analysis to explain variation in students Civic Knowledge, Italian language and Mathematics for Italy • M3 • the negative effect cannot be attributed to higher levels of autonomy • In fact: • only the linear term shows a statistical significant effect (β = - 0.109, se = 0.035) • the effect of the quadratic term is almost nil (β = 0.006, se = 0.013)

13. Discussion and conclusions For Italy: higher levels of school autonomy are associated with lower student achievement irrespective of the subject in which achievement was measured. • Possible leads for interpreting results: • Especially in elementary and middle school, the autonomy is only partial: • Organizational and didactic • NOT really financial: limited funding diversification, constraints to use of (human) resources • Contradictions of a low school autonomy, but an even lower accountability: to the point mostly bureaucratic accountability, little emphasis on learning outcomes  scarcity of feedback processes to school improvement Limitations and steps forward: Findings should be further explored by looking at different effects of different areas of decision-making as well as measures of school accountability.

14. Conclusions and future research • Especially in the first phase of education, we would be confronted with a mere partial view of all the available decision-making levers • Contradictions of a school autonomy "in midstream” • Contradictions of a low school autonomy, but an even lower accountability!

15. Thank you for your attention