ECER Conference, Vienna 29 th September 2009 Matt O’Leary University of Wolverhampton, UK

1. Playing the game of panoptic performativity? Perspectives on the grading of observations of teaching and learning in Further Education colleges in England? ECER Conference, Vienna 29th September 2009 Matt O’Leary University of Wolverhampton, UK moleary@wlv.ac.uk

2. Contents of presentation • Focus of the presentation • Synopsis of the research design • Overview of the FE context • Emergent themes from the data • Theoretical lenses • Preliminary conclusions

3. Focus of the presentation • An exploration of the use of graded observation of teaching and learning (OTL) in Further Education (FE) colleges in the West Midlands region of England & its impact on the professional lives and identities of those working in the FE sector

5. Location, context, population size & sample: 10 FE colleges from a population of 22 across the West Midlands region of England 3 different stakeholder groups involved: tutors/teaching staff, middle managers & senior management team (SMT) 50 participants per college (total n = 500) Research methodology: Methodological framework – ‘constructivist-interpretive’ Mixed methods – quantitative & qualitative tools of inquiry Phase 1: 50 questionnaires to each college (total n = 500) Phase 2: 21 interviews in 3 selected colleges (total n = 21) Combination of purposive & stratified random sampling Synopsis of the research design

6. Overview of FE context • 1990s → introduction of graded OTL as an initiative on a widespread scale in the FE sector • 1999 → emergence of graded OTL as an important initiative in the ‘drive for excellence’ aimed at raising standards and improving the quality of T & L can be linked to 2 key policy developments: • Introduction of Further Education National Training Organisation (FENTO) teacher standards (1999) & mandatory qualifications from Sept 2001 • New inspection framework - Ofsted & ALI working alongside each other from 2001 before eventually merging into one inspectorate in 2007

7. Emergent themes • Grading as an “ego-involving” form of feedback which leads to implicit & explicit “labelling” : ‘The grading of observations is divisive – we are given tables of how many people got which grade – it has almost become unhealthy competition – it’s unnatural too. Personally I hate the process though I get good grades. I live in fear of failing next time.’ (Pam – Q) ‘The people who got the grade ones were mainly embarrassed to be named and they didn’t want people to know so straight away they have taken away their right of confidentiality by publishing their names. People who hadn’t got a grade one like myself, then felt somehow inferior to these people.’ (Sue – I)

8. Emergent themes • Grading as an “ego-involving” form of feedback which leads to implicit & explicit “labelling” : ‘On a staffroom basis, I think it leads to a process of labelling of people to a certain extent i.e. who got a grade 1, who got the lowest grades, etc, even though people know it just represents a snapshot, there is a little bit of competition that can sometimes lead to division amongst some staff. I think people attach quite a lot of significance to it in a way, which is a little ironic because people don’t necessarily respect the whole thing because it’s imposed and we have no input. And, at the same time they seem to care a lot what grade they’re given. (Elizabeth – I) ‘Well, the people who are given a grade one are persecuted. They have lots of people sent to them to observe their lessons as good practice, they’re asked to do staff training, etc. Some people are a bit tongue in cheek saying they’d rather have a grade two because then they won’t get the hassle, but you do hear that on a regular basis really.’ (Tony – I)

9. Emergent themes • Credibility, reliability & inconsistency of measurement: ‘The 1st review I felt as happy as Larry because I got a grade 1, the 2nd observation I got a grade 3, verging on a grade 4. To be fair, I don’t believe my teaching had changed so much in the space of a year. In fact, I did pretty much the same type of lesson! The two observers’ opinions on what constitutes a good lesson were so different that I didn’t know who to believe. There’s got to be something wrong if a teacher can come out with a grade 1 one year and a grade 3, almost a 4, the following year.’ (Bob – I) ‘I have two Programme Managers for one post but job shared and one thinks grade ones are acceptable and the other one thinks no one should get grade ones because that would be perfection and no one’s perfect as we’ve all got room to improve and I just thought well I would like to know how that then makes the process valid because if you’ve got two different measuring sticks, how can that validate what you’re doing?’ (Eve – I)

10. Emergent themes • Ability & suitability of the observer: ‘The observation process is successful if the observee has confidence in the abilities of the observer.’ (Brian – Q) ‘The observer must be a current practitioner themselves in the subject area and understand the context of the session, content and level of the learners involved. Otherwise the whole process is flawed.’ (Jackie – I) ‘Observations are great if they are carried out by a person qualified to do the job and who understands the implications of their comments and actions.’ (Ghania – Q)

11. Emergent themes • Use of grading as a “disciplinary power”: ‘At the end of the year in our self assessment report, we will report on the number of ones, two’s, threes and fours and I think it’s basically worthless but it’s something that all colleges do at the moment. Our policy on lesson observations is that every teacher will be observed twice in the academic year. The second observation is waived if the teacher achieves a grade one or grade two in the first observation. The big tension is the threes because the three gets you, “Oh no, not one of those mentors!” They get a grade three and they get allocated a mentor so it’s almost like the mentor has become the punishment and this worries me. ‘ (Steve – I) ‘You have to get a 3 or above or you won’t get your salary increment for that year.’ (Sue – I)

12. Theoretical lenses • Foucault (1977; 1980) • Gramsci (1971)

13. Theoretical lenses - Foucault • Metaphor of the Panopticon • Disciplinary power • Foucault’s notion of the “examination” as ‘a normalising gaze, a surveillance that makes it possible to qualify, to classify and to punish’ (1977: p. 172). • Foucault (1980) - relationship between power and knowledge • Dominant discourses & regimes of truth

14. Theoretical lenses - Gramsci • Hegemony & its usefulness in understanding the concepts of ideology and power > links to Foucault • Relevance of hegemony to learning and professional development • Contested & dynamic nature of hegemony

15. Preliminary conclusions • Normalisation of observation in many colleges but it still remains a contested terrain • Tangible effects on notions of self-efficacy, self-confidence, self-esteem, motivation & morale of staff – positive & negative • Contrasting means of making sense of this mechanism & engaging with it amongst staff in the sector • ‘Successful’ staff rewarded with a greater sense of freedom & ‘higher status’, whereas failure calls forth closer surveillance – support in some instances, neglect & labelling in others

16. Preliminary conclusions • Concerns about the inconsistency, reliability & subjectivity of grading & feedback amongst observees casts doubt on the underpinning rationale that graded OTL help to inform future areas of curriculum & staff development • Evidence of ‘playing the game’ from all 3 stakeholder groups & ‘ticking boxes’ • Observation valued by all as a tool for development but there are caveats attached to its value i.e. purpose, what happens to data, ownership and/or involvement in the process, time & resources allocated