Evidence of Q uality and E quity in E ducation

1. Evidence of Quality and Equity in Education Péter Radó Belgrade, 06.12.2011.

2. A sample evidence 1.

3. A sample evidence 2.

4. A sample evidence 3.

5. A sample evidence 4.

6. A few questions:(Assuming that these data are available ) • Is there a demand for these data among the key players of education? • Do you have access to the information allowing the interpretation of these data? What would it tell about the quality of educational services? • Do you have the capacity to transform the conclusions drawn from these data to actual design of policies and implementation strategies? • Can you imagine a minister of education explaining this statistical analysis in a 30 seconds TV interview? • Can you imagine convincing middle class parents about mixing their children with disadvantaged children on the basis of this analysis? • Can you imagine convincing teachers about forming heterogeneous classrooms instead of forming easily taught homogeneous classrooms? • Can you imagine municipalities reorganizing their local school systems in order to ensure equal distribution of disadvantaged students on the basis of this analysis?

7. Questions emerging from the questions • What can generate demand for evidence? • How to produce evidence? • How to determine quality and equity relevant evidence needs? • How to ensure the interpretation of evidence? • What are the traps to be avoided? • What is the best use of evidence?(Influencing the quality of the policy discourse.)

8. Generating demand for evidence From dumb centralization to intelligent post-bureaucratic governance

9. Generating demand for evidence: governance and management Making education a complex adaptive system by: • making schools autonomous learning organizations accountable for the quality of their services and the equity of their outcomes, • decentralization, deregulation, procedural regulation, • mandatory multilevel short- and medium term planning, • Intelligent professional accountability systems with strong focus on educational outcomes produced by whole schools, • ensuring the openness of decision making at all levels.

10. Generating demand for evidence: policy-making The normative approach to EBPM ↔ the very contextual nature of quality and equity→ „Intelligent policy making” Learning by open deliberation: enrichment of reasoning with the tacit and practical knowledge of practitioners + evidence informing the actors of the policy discourse

11. Information production in education 1. Quality evaluation embedded to a performanace management system • Connecting goals/targets with quality evaluation instruments • Student performance assessment informing (external and internal) school evaluation • Connecting the pillars of by quality evaluation by an integrated indicator/reporting system • Strengthening accountability by intervention on the basis of quality evaluation information • Balancing accountability and organisational learning in schools

12. Information production in education 2. The only reliable measure of strengths and weaknesses is international comparison. → Enabling information systems for international referencing (EU OMC indicators and benchmark, OECD „Education at a glance” indicators, participation in international student achievement surveys, connecting internal and international assessment systems, etc.) → Ensuring the contextual relevance of international comparative information by further analysis and comparative research (See: European educational performance patterns)

13. Determining quality and equity related information needs Connecting the results of meta-evaluation with the instruments of quality evaluation

14. Interpretation: research and communication • Research for deeper understanding: e.g. what makes heterogeneous schools more capable to compensate for disadvantages? • Typical problem: research substituting information production versus research for the interpretation of information produced on a regular basis. • Communication: transforming evidence to clear policy messages. → „Simplexity” (Ora Ito) E.g. „Children in schools with heterogeneous intake develop better.”

15. A few possible traps • The trap of measurement: what we measure becomes a problem, what we don’t measure remains invisible and ignored • „Economism”: considering the indicator equal with the indicated fact, policies aiming at „improving indicators” instead of solving problems • Growing complexity → the fear of loosing control → escaping from complexity by maintaining (or returning to) centralization created simplicity • Striving for the „optimum” instead of striving for the „possible” (ignoring power, influence, prejudice, whim, etc. by focusing only on evidence) • Evidence as legitimacy substitute

16. Finally…What determines the quality of the policy discourse:the technology and references of policy-making What has changed in Hungary between 2010 May and July? Politics, nothing else.