Researching and Evaluating innovations in schools and systems with a special focus on the use of information technology

1. Researching and Evaluating innovations in schools and systems with a special focus on the use of information technology John Schostak Institute of Education Manchester Metropolitan University j.schostak@mmu.ac.uk This powerpoint may be found at: http://www.enquirylearning.net/ELU/Issues/Information%20Technology/taiwan.ppt.htm

2. Purpose • To describe the ‘innovation/change problematic’ with reference to three case studies giving insights in: • The micro-level of action • The system level of a nation • The ‘global-local level’ of distributed ‘communities’ • To provide insights into the process of researching and evaluating educational contexts

3. Introduction: A first lesson in what not to do. • 1981-1984 - computers in schools. Learning the need: • for all decision makers to have a broad philosophical or conceptual understanding of the purpose of an innovation • to understand the nature of the social system or organisation in which the innovation is to be implemented • for dialogue with practitioners • for professional development appropriate to the innovation • for appropriate resourcing

4. Closed systems the desire for mastery or control Often thought of as an ideal and leading to the ‘gold standard’ of RCTs; or, of ‘audit trails’ Open systems complexity and uncertainty produces the need for political and ethical strategies Often thought of as ‘practical realities’ requiring qualitative research designs The Innovation problematic versus In the realities of day to day practise, research and evaluation focuses on the tension between the closed and the open and the points of conflict and struggle that are produced in essentially complex, uncertain, frameworks of social interaction.

5. The ideal structure (closed) From: Schostak, J.F. (2002) Understanding, Designing and Conducting Qualitative Research in Education. Framing the Project. Open University Press

6. The ‘real’ structure: a stealth architecture Competing interest groups Alternative practices Different resourcing Unpredictable outcomes From: Schostak, J.F. (2002) Understanding, Designing and Conducting Qualitative Research in Education. Framing the Project. Open University Press

7. 3 questions • Micro-level (school): how to get people involved, empowered and so generate and embed cultural and organisational change? • Macro-level (State/system): how to set the agenda, provide the vision and catalyse change across a system? • Local-global level (on-line, distributed communities): how to reconceptualise educational practices in communities that are an interface between ‘real’ (local) and ‘virtual’ (global, distributed) spaces or ‘places’?

8. http://www.enquirylearning.net/ELU/Issues/Education/archivesEarlyyears.htmlEarly years talking and listening project: 1988-9 • Expression by each teacher of their key educational values and philosophies • Whole school (teachers, teaching assistants, kitchen staff, caretaker, pupils) approach to reflecting on and researching practice - broadly Action Research employed as a mechanism/procedure to formulate and guide change • Material architecture and equipment of the school reconceptualised in terms of how it could support the new evolving practices • Outcomes: • Culture and ethos of the school changed • Innovation embedded in new professional practices • School radically revitalised

9. Linking aims, approach and evidence • To encourage and develop children's responsibility for their own actions and behaviour. How? The procedure involved: • recognition of problem/dispute • making the problem/dispute explicit • individuals taking responsibility for the solution to their own problems/disputes - an adult may help to ensure 'turn taking' so that all views are heard • generating possible solutions • agreeing a solution • ensuring no one is upset by a solution • keeping to the solution unless a further problem arises • Evidence: through Action Research, videos, observations, interviews were carried out, analysed and used as a basis for policy development as well as professional development

10. Evaluation of Finnish National Curriculum 1995 • A senior Education Board official: when trying to define the inspiration for the reforms pointed not to key facts and practices but to key values which were, for him, summed up as: Truth, Beauty, Goodness. (fieldnotes 14/2/95) • The Framework Curriculum for the Comprehensive School 1994, stated: ‘all human solutions are connected with values’ and that the ‘balanced development of physical, psychic and social resources makes it necessary for us to bring up questions which have to do with our health and well-being’ (p.10).

11. For many but not all schools: As pointed out in our evaluation report: The head teacher of a large project school explained to us that they had gone through the two year process, had found it tiring but useful, but now that the project was finished, they had returned to their traditional methods as these were more efficient and easier to manage.

12. CIEL Project 2003-4: the emergent global-local community • Collaboration In E-Learning (CIEL) • DfES funded Forum Trust, £200,000 • 5 case studies of the challenges involved in sharing practice across organisations • one of these, a study of the use of a Tiki in a primary school. A Tiki is an open source content management system ideal for creative collaborations The case became an example of the material environment problematising both philosophy and practice

13. The on-line environment

14. Some Early Implications: • The new environment requires different pedagogical and learning strategies to those of the traditional classroom because: • Hierarchical structures are replaced by horizontal structures • No central control • Order ‘emerges’ rather than imposed • Children/learners are active agents, decision makers • The boundaries between classroom, home and world dissolve: the new environment is anywhere, anyplace, anytime • New communities of participation and interest emerge: parents, teachers, researchers, policy makers, creative practitioners …. etc Does this imply the need for micro and macro innovations like those implemented by the previous two case studies?

15. Back to the future • Conceptual changes are required: hence, the re-emergence, by stealth, of child centred, ‘progressive education’ under the name of personalisation • New practices are required: hence, ‘breaking into the curriculum’ or ‘hacking’ as a means of developing the kinds of communities of practice or ‘intelligence communities’ implicit in the e-Park concept

16. E-Park - the infrastructure

17. Three conclusions for research and evaluation • For effective change and desired outcomes identify and ensure coherence of the dynamic relations between: • The conceptual frameworks held by competing communities of interest • The alternative practices favoured by each community of interest • The organisation of material resources

18. Decision making under conditions of complexity and uncertainty will involve all actors in: • Political decisions, i.e. any decision that advantages one group will disfavour or threaten others • Ethical decisions concerning justice, ‘proper’ behaviour, and the ‘good society’ - different communities have different traditions, values, belief systems Hence: formative structures for the negotiation of values and the agreeing of decisions are required

19. To deal with the complexity and uncertainty during change, research and evaluation must be integrated at every level to • inform the decision making of all interested parties by identifying problems and opportunities • identify the range of alternative rationales for action and the deployment of resources • explore the consequences of choosing one course of action rather than another • develop approaches to professional development • embed cultural and institutional change throughout systems Hence, research and evaluation are not peripheral to decision making and the implementation of innovations. They are fundamental to their success

20. References Archer, M., Bhaskar, R., Collier, A., Lawson, T., and Norrie, N., (eds) (1998) Critical Realism. Essential Readings, London and New York: Routledge Bhaskar R. (1975 and 1978) A realist theory of science, Hassocks, Sussex ; Harvester Press Dewey, J. (1938) Experience and Education, New York, Collier Freire, P. (1973) Education: The Practice of Freedom, London, Writers and Readers Publishing Cooperative. Hargreaves, D. H. (2003) ‘Education Epidemic: Transforming Secondary Schools through Innovation Networks’, London, Demos Holt, J. (1969) How Children Fail, Harmondsworth, Penguin Illich, I. (1971) Deschooling Society, London: Calder and Boyers Norris, N., Aspland, R., McDonald, B., Schostak, J.F. & Zamorski, B. (1995) An Independent Evaluation of Comprehensive Curriculum Reform in Finland, CARE, UEA; Finnish Board of Education Pawson, R., and Tilley, N. (1997) Realistic Evaluation, London; Thousand Oaks, California: Sage Rogers, C. (1969, 1983) Freedom to Learn, revised as: Freedom to Learn for the 80s, Columbus, London: Merrill Sayer, A. (1993) Method in Social Science. A Realist Approach, London, New York: Routledge Schostak, J. F. (ed.) (1988) Breaking into the Curriculum: the impact of information technology on schooling, London, New York. Methuen Schostak, J.F. (2002) Understanding, Designing and Conducting Qualitative Research in Education. Framing the Project. Open University Press Schostak, J. F. (2002) 'The gobalisation of Education - what are the challenges?', Information Technology Strategy Committee Seminar Series, Hong Kong Institute of Education Smith, J., Schostak, J., Phillips, E., Hough, M., Fleet, K., Davies, B., and Brewer, L. (2004) Collaboration in E-Learning (CIEL). A report for the Department of Education and Skills, Norwich, Forum Trust Stenhouse, L. (1975) An Introduction to Curriculum Research and Development, London: Heinemann