Graded Lesson Observations in FE Colleges: Winners and Losers

2. Playing the game of panoptic performativity? Who are the winners and losers of lesson grading in classroom observations? School of Education Conference 6th May 2009 Matt O’Leary

3. Contents of presentation • Defining the object of study • Overview of the FE policy backdrop • Overview of the research methods • Theoretical lenses • Emergent voices • Preliminary conclusions • References

4. Defining the object of study • An exploration of the use of graded lesson observation schemes in internal quality reviews in FE colleges in the West Midlands & the impact that they have on the professional lives and identities of those working in the sector

5. Overview of FE policy backdrop • 1990s → introduction of observation as an initiative on a widespread scale in the FE sector compared to the middle of 19th century in schools (Grubb, 2000; Wilcox & Gray, 1996) • 1999 → 2 key policy developments triggered the use of observation as a key driver in a wider policy-practice agenda of workforce reforms aimed at raising standards and improving the quality of T & L: • Introduction of Further Education National Training Organisation (FENTO) teacher standards (1999) & mandatory qualifications from Sept 2001 • New inspection framework - Ofsted & ALI working alongside each other from 2001 before eventually merging into one inspectorate in 2007

6. Overview of FE policy backdrop • Ofsted’s grip over FE policy tightens- leading role in directing the policy agenda of quality improvement in the sector, as evidenced by some of the key publications to emerge in recent years that have influenced FE (e.g. Ofsted, 2003; 2004a; 2004b). • Why Colleges Fail (2004a) & Why Colleges Succeed (2004b) identify graded observation schemes as a cornerstone in the QA & self-assessment process in colleges & seek to valorise them: ‘In some colleges there is a reluctance to grade observations which are viewed as developmental rather than quality control activities’. Ofsted’s view is that such schemes are ‘insufficiently robust, partly because the observations are not graded’ (2004a: p. 14).

7. Overview of FE policy backdrop • Recent developments (2007): • Ofsted introduces a new carrot & stick regime of: • ‘proportionate’ & ‘light touch’ inspections for most successful colleges & greater role for self assessment • ‘more robust’ intervention strategies for those underperforming & inadequate colleges • Consequences = blanket adoption by college SMTs of Ofsted’s assessment framework (i.e. 4 point grading scale & criteria) as part of their internal observation schemes

9. Location, context, population size & sample: 10 FE colleges across the West Midlands Tutors/teaching staff, middle & SMTs 50 participants per college (total n = 500) 3 different stakeholder groups involved: Overview of the research methods

10. Overview of the research methods • Sampling strategies & research tools: • Combination of purposive & stratified random sampling techniques used • Mixed methods – quantitative & qualitative tools of inquiry • Phase 1 – 50 survey questionnaires circulated across all 10 colleges (total n = 500) • Phase 2 – 21 interviews carried out in 3 selected colleges (total n = 21)

11. Theoretical lenses – Who or what am I?

12. Theoretical lenses - Foucault • Graded observation as a mechanism of ‘disciplinary power’ i.e. power exercised on ourselves by ourselves through self-surveillance → links into recent & future movements towards ‘proportionate’ & ‘light touch’ inspections • Clear links between the application of Ofsted’s framework of assessment for graded observations of T & L and Foucault’s notion of the examination as ‘a normalising gaze, a surveillance that makes it possible to qualify, to classify and to punish’ (1977: p. 172). Teachers are categorised & differentiated by means of the 4 point scale

13. Theoretical lenses - Foucault • Foucault’s work (1980) also highlights the relationship between power and knowledge. Knowledge is socially produced by a number of connected mechanisms that ‘determine how knowledge is accumulated by prescribing correct procedures for observing, researching and recording data and for disseminating the results of investigations’ (Brookfield, 2005: p. 137) – Those with the greatest command of these mechanisms are able to create ‘dominant discourses’ and ‘regimes of truth’ e.g. ‘robust systems’ & ‘solid quantitative evidence’

14. Theoretical lenses - Gramsci • Concept of ‘hegemony’ i.e. the way in which people are convinced to embrace dominant ideologies (in this case Ofsted’s & SMT’s framework for grading observations) as always being in their own interests based on ‘common sense’, but these ideologies actually work against them e.g. Perryman (2007) claims that inspections distract teachers from teaching & being creative

15. What is ‘panoptic performativity’? • Panoptic – relating to Bentham’s panopticon • Perfomativity – term first used by Lyotard (1984) to suggest that postmodern society is obsessed with efficiency & effectiveness & that efficiency is increasingly ‘measured according to an input/output ratio’ (1984: p. 88). • The concepts of ‘performativity’ and ‘accountability’, often associated with the work of Ball in the field of education (1995, 2003), can be seen as products or consequences of the introduction of market values into the public services and the desire to apply the principles and practices of the world of business to the education sector • ‘Panoptic performativity can describe the experience of an inspection regime in that teachers and [students] feel as if they are constantly being observed and perform accordingly in order to escape the regime’ (Perryman, 2007: p. 155)

16. Emergent voices • Inconsistency & subjectivity of grading and feedback is a concern raised amongst participants. What one observer regards as ‘effective practice’ may be totally different in some cases to others. Evidence from the schools’ sector in relation to the difficulty that Ofsted has experienced in agreeing on what constitutes effective teaching would also seem to corroborate such a standpoint (Gilroy & Wilcox, 1997; Richards, 2001; Wragg, 1999). CC02 comments: • ‘The 1st review I felt as happy as Larry because I got a grade 1, the 2nd observation I got a grade 3, verging on a grade 4. To be fair, I don’t believe my teaching had changed so much in the space of a year. The two observers’ opinions on what constitutes a good lesson were so different that I didn’t know who to believe. There’s got to be something wrong if a teacher can come out with a grade 1 one year and a grade 3, almost a 4 the following year. I think it shows you that someone you don’t know coming in from outside is not really in a good position to be making an evaluation of your teaching.’

17. Emergent voices • Repercussions of grading - There was evidence to suggest that graded observations could lead to a process of labelling of teachers as ‘grade 1, 2, etc’. In turn this could lead to divisions amongst staff and unrest in staffrooms. For example, DC40 comments that: • ‘The grading of observations is divisive – we are given tables of how many people got which grade – it has almost become unhealthy competition – it’s unnatural too. Personally I hate the process though I get good grades. I live in fear of failing next time.’ • The demands on teachers to produce a grade 1 lesson are succinctly encapsulated in the comments of MC25: • ‘To achieve a high grade for an observation you need to produce an ‘all singing, all dancing’ lesson and be constantly upbeat with students. Lessons are, in some part, like this but to keep up the sort of momentum, tempo, paper work and constant buzz in class at all times would be exhausting.’

18. Emergent voices • SO41’s comments reveal an interesting perspective on the effects of preparing for graded observations: • ‘I feel that they do not represent the general quality of teaching. I also feel that teaching for other classes the teacher has suffers in the lead up to an observation because of the planning taking place for the observed lesson and due to exhaustion after it!’ • CC04 talked very candidly about the labelling effects of the grading exercise amongst her colleagues, and this is bearing in mind that she was given a Grade 1 for her observation: • ‘On a staffroom basis, I think it leads to a process of labelling of people to a certain extent i.e. who got a grade 1, who got the lowest grades, etc, even though people know it just represents a snapshot, there is a little bit of competition that can sometimes lead to division amongst some staff. I think people attach quite a lot of significance to it in a way, which is a little ironic because people don’t necessarily respect the whole thing because it’s imposed and we have no input. And, at the same time they seem to care a lot what grade they’re given.

19. Emergent voices • The comments of observer MC43 highlight the importance of notions of trust and confidence in the observer-observee relationship: • ‘I feel that observations are an integral part of the review process and need to be managed carefully to ensure trust and confidence is established and maintained between staff and line managers.’ • DC38’s comments emphasise how positive and beneficial the experience can be if the observer is deemed to be suitably qualified to carry out their role: • ‘The observation process is successful if the observee has confidence in the abilities of the observer, and also confidence in the training that has been experienced, prior to the observation. In my experience the observation should only ever be a supportive process.’ • Similarly, DC15’s comments echo these thoughts: • ‘Observations are great if they are carried out by a person qualified to do the job and who understands the implications of their comments and actions. Peoples’ careers and future well being are at stake and this very important part of our life should not be treated flippantly. We should treat this as serious time well spent.’

20. Preliminary conclusions • Normalisation of observation in many colleges but it still remains a contested terrain. The ‘normalising gaze’ classifies staff according to their grade & there is evidence of ‘punishment’ for those that underperform • Tangible effects on notions of self-efficacy, self-confidence, self-esteem, motivation & morale of staff – positive & negative • Contrasting means of making sense of this mechanism & engaging with it amongst staff in the sector • Teaching staff who are defined as ‘successful’ by management are rewarded with a greater sense of freedom, whereas failure calls forth closer surveillance – support in some instances, neglect & labelling in others

21. Preliminary conclusions • Some concerns about the inconsistency & subjectivity of grading & feedback amongst observees • Quantiative data set helps college SMTs & HoDs identify ‘outstanding’, ‘good’, ‘satisfactory’ (but improving?), ‘satisfactory’ (but not improving?) & ‘inadequate’ practice to inform future areas of curriculum & staff development • ‘Muddying of the waters’ – switching gears from one use of observation to another with the same ‘tool’ i.e. quality review, teacher education/development, peer observation, curriculum development

22. References • Ball, S. (1993) Education Markets, Choice and Social Class: The Market as a Class Strategy in the UK and the USA. British Journal of Sociology of Education, Vol. 14, No. 1, pp. 3-19. • Ball, S.(2003) The teacher’s soul and the terrors of performativity. Journal of Education Policy, Vol. 18, No. 2, pp. 215-228. • Boardman, A. G. & Woodruff, A. L. (2004) Teacher Change and ''High-Stakes'' Assessment: What Happens to Professional Development? Teaching & Teacher Education: An International Journal of Research and Studies, Vol. 20 No. 6, pp. 545-557. • Brookfield, S. D. (1995) Becoming a Critically Reflective Teacher. San Francisco, CA: Jossey-Bass. • Brookfield, S. D. (2005) The Power of Critical Theory for Adult Learning and Teaching. Maidenhead: Open University Press. • Foucault, M. (1977) Discipline and Punish – The Birth of the Prison. Harmondsworth: Penguin. • Foucault, M. (1980) Power/Knowledge: Selected Interviews and Other Writings, 1972-1977. New York: Pantheon Books. • Freire, P. (1977) Pedagogy of the oppressed. New York: The Continuum Publishing Corporation.

23. References • Freire, P. (2005) Teachers as Cultural Workers – Letters to Those Who Dare Teach. Cambridge, MA: Westview Press. • Gilroy, P. & Wilcox, B. (1997) OFSTED, Criteria and the Nature of Social Understanding: A Wittgensteinian Critique of the Practice of Educational Judgement. British Journal of Educational Studies, Vol. 45/1, pp. 22 – 38. • Gramsci, A. (1971) Selections from the Prison Notebooks of Antonio Gramsci – edited & translated by Quintin Hoare & Geoffrey Nowell Smith. Lawrence & Wishart: London • Grubb, W. (2000)Opening Classrooms and Improving Teaching: Lessons from School Inspections in England. Teachers College Record, Vol. 102, No. 4, pp. 696-723. • Hatton, P. (2008) ‘Observing teaching is central to improvement.’ The Talisman, Issue 70. London: Ofsted Publications Centre. • Jeffrey, B. and Woods, P. (1998) Testing Teachers: the effect of school inspections on primary teachers. London: Falmer Press. • Lyotard, J-F. (1984)The Postmodern Condition: a report on knowledge, vol. 10. Manchester: Manchester University Press. • Montgomery, D. (2002) Helping Teachers Develop through Classroom Observation, 2nd Edition. London: David Fulton Publishers.

24. References • Ofsted1762 (2003) The initial training of further education teachers: a survey. Available at www.ofsted.gov.uk/publications/index. • Ofsted 2408 (2004a)Why Colleges Fail. London: Ofsted Publications Centre. • Ofsted 2409 (2004b) Why Colleges Succeed. London: Ofsted Publications Centre. • Ofsted 2651 (2006) A Handbook for Inspecting Colleges. London: Ofsted Publications Centre. • Perryman, J. (2006) Panoptic performativity and inspection regimes: disciplinary mechanisms and life under special measures. Journal of Education Policy. Vol. 21, No. 2, pp. 147-161. • Richards, C. (2001) School Inspection: a critique. Northampton: Philosophy of Education Society. • Tilstone, C. (1998) Observing Teaching and Learning – Principles and Practice. London: David Fulton. • Wilcox, B. & Gray, J. (1996) Inspecting Schools – holding schools to account and helping schools to improve. Buckingham: OUP. • Wragg, E.C. (1999) An Introduction to Classroom Observation, 2nd Edition. London: Routledge.