Evaluation practice in the Nordic countries: Different national traditions or a common approach?

1. Evaluation practice in the Nordic countries: Different national traditions or a common approach? Hanne Foss Hansen Department of Political Science University of Copenhagen

2. Structure • Educational evaluation: Concepts and approaches • Case 1: Higher education -Brief reviews country by country -Similarities and differences • Case 2: Primary and secondary education (P/S) • The effects of all this evaluation? The future?

3. The concept of evaluation • Everyday language: Measurement, assessment, judgement • Evaluation language: ”A careful assessment of the merit and worth of processes, structures, output and outcome of interventions and organizations, intended to play a role in future, practical actions situations”

4. The concept of educational evaluation • Testing, student assessment, programme evaluation, personel evaluation, auditing, accreditation, benchmarking, curriculum evaluation and probably even more.

5. Educational evaluation: Focus on many levels -Individuals (pupils, students, teachers) -Classrooms/courses -Curriculum/programmes -Organizations (schools, universities) -Fields (all schools in a municipality, all programmes in a discipline) -The national level (national quality development and quality assurance systems) -The international level (PISA, EQUIS in the business school area)

6. Educational evaluation: Many purposes, many uses -Documenting -Controlling -Learning/improving -Reforming -Legitimating -Symbolizing

7. Focus today primarely on • The new forms of evaluation (programme evaluation, auditing, accreditation etc.) not on the classical questions of testing and student assessment • Meso-evaluation defined as evaluation coupled not only to professional practice but also to educational policy

8. Higher education I • Adoption of evaluation in the late 1980´s • 1992-1999: The Danish Center for Evaluation of Higher Education • 1999: The center is reorganized into the Danish Evaluation Institute (EVA)

9. Higher education II • 1990’s: Programme evaluation • 2002: Accreditation is introduced • 2003: A new university law stresses the responsibility of the universities themselves to conduct evaluations (EVA unclear role) • 2004: Auditing is introduced ---- • 2005: EVA is made responsible for accreditation of professional education

10. Higher education I • Adoption of evaluation in the late 1960’s and early 1970’s, gaining renewed priority in the mid 1990’s • 1995 The National Agency for Higher Education (Högskoleverket) is established

11. Higher education II • 1999-2002: Auditing is the main task • 2001-2006: Programme evaluation becomes the main task • Accreditation is also part of the picture

12. Higher education I • Adoption of evaluation in the mid 1990´s • 1996: The Finnish Higher Education Evaluation Council (Finheec) is etablished

13. Higher education II • As law places responsibility for evaluation with the higher educational institutions an important purpose of the council is to help institutions to develop quality assurance and development systems • The council also initiates evaluations of different types • Accreditation is important in relation to polytechnics and professional courses • 2004: Auditing

14. Higher education I • Adoption of evaluation in the late 1990´s • 1998: ”Norgesnettrådet” is established • 2003: The Norwegian Agency for Quality Assurance in Education is established

15. Higher education II • Auditing of all higher educational institutions • Accreditation of programmes and institutions applying for new programmes and institutional status • The Ministry of Education initiates evaluations of higher educational reforms (Høgskolereformen, Kvalitetsreformen)

16. Higher education • Adoption of evaluation in the mid 1990’s • 1999: It becomes mandatory for higher educational institutions to develop quality assurance systems • The Ministry of Education initiates programme evaluation ad hoc • No formalised accreditation system

17. Similarities across countries in talk, organisation and focus • Adoption of meso-evaluation in all countries • Anchoring evaluation in semi-autonomous organizations specialized in evaluation (not Iceland) • Educational evaluation is decoupled from evaluation of research • A turn towards auditing (N, DK, FIN)?

18. From national imitation to international regulative pressures? strong Bologna 2005 National pressures strong weak 1990 weak International pressures

19. Factors explaining convergence • Public-sector reforms: New Public Management, focus on results and effectiveness • Internationalization: The Bologna proces and the aim of establishing a European Higher Education Area in 2010 • Networking across agencies at Nordic as well as European level

20. Differences in institutional processes • Time span in adoption (from Sweden in the late 1960’s, to Denmark in the late 1980’s and Norway in the late 1990’s) • Time span in institutionalization (e.g. routinization in Denmark from 1992, in Norway from 2003) • Norway as the late adopter has constructed the most radical system

21. Differences in balances between quality development (QD) and control (C) purposes • DK: QD more than C (except professional education) • S: From C more than QD to QD more than C • FIN: QD more than C (except professional education) • N: C but also QD • IS: QD more than C

22. Differences in decision contexts • From Denmark where there is no direct coupling to sanctioning and rewarding (except in professional education) to Norway where there is a direct coupling to sanctioning and rewarding with Sweden somewhere in between

23. Differences in evaluation models - Self-evaluation is an important element in DK, S and FIN but not in N

24. Differences in composition of evaluation panels

25. Differences in coordination across individual evaluations

26. Factors explaining divergence Differences in: • political-administrative cultures • strategies in public-sector reforms • structures and traditions in educational systems • timing and content of higher educational reforms

27. P/S education • Late 1990’s the Ministry of Education introduces a program ”Quality development in public Schooling” (attention and tools ) • 1999: EVA gets responsibility for evalution in P/S • 2002: A law about transparency and openness makes it compulsory to educational institutions to publish evaluations of the quality of teaching • 2005: Government proposes to establish a council and an agency for quality development

28. P/S education 1997: Municipalities have each year to work out written quality reports 2003: The agency for education is split up in the Swedish Agency for Education and an agency for school development 2004-2009: Inspection programme. Inspection reports serve as starting points for improving the quality of municpal schooling.

29. P/S education • 2003: A council for educational evaluation is established. The council has to plan and implement external evaluations as well as develop methods and coordinate local evaluation

30. P/S education • 2004: The Directorate for Primary and Secondary Education is established. The directorate is responsible for an internet-based quality assessment system ensuring transperency in quality information.

31. P/S education • Schools have to do and publish self-evaluations • Every 5th year The Ministry of Education assesses the evaluation methods used by schools (site-visits)

32. P/S education: Similarities • Evaluation adopted in all countries • International studies have put educational quality and evaluation on the agenda (PISA & TIMMS) • All countries build national institutional capacity to deal with quality and evaluation (increasing state control) • Transparency in monitoring is important (strenghtening market forces)

33. P/S education: Differences • Balances between quality development and control purposes (S: C control but also QD; DK, N, FIN and IS: more soft approaches) • ? – Too early to really conclude on the practice of the new agencies

34. Higher education -Time span in adoption (from late 1960’s to late 1990’s) -Policy-driven development P/S -Later adoption but no time span -Problem-driven development (DK, N) Comparing the two cases

35. Evaluation practice in the Nordic countries: Different national traditions or a common approach? Conclusions • Similarities in talk • Similarities and important differences in actions • Evaluation is an elastic concept giving room for national and local constructions

36. Effects of growth in meso-evaluation I Two very different ways of thinking: • Optimism related to the development of learning organizations and a knowledge society • Pessimism related to the development of an audit society based om distrust

37. Effects of growth in meso-evaluation II • Are educational institutions transformed into learning organizations or into ”auditable commodities”? • Is professional practice part of or de-coupled from evolving evaluation cultures? Limited empirical knowledge in the Nordic countries

38. The future A turn towards: -Auditing and accreditation? -Evidence-based professional practice? -Evidence-based educational policy?