ATTITUDINAL SURVEYS AND SELF EVALUATION

1. ATTITUDINAL SURVEYS AND SELF EVALUATION JUNE 2011 London Conference Geoff Davies

2. What are schools about? Effective Learning Opening windows on a world Choices Realising potential Achievement and Standards Qualifications

3. BUT DO YOU RECOGNISE THIS??? • You arrive at school one morning and you receive the following information • Two pupils have misbehaved on the bus upsetting a third child • A parent has phoned saying her child is being bullied in school and refuses to come to school • A pupil has told her LSA that her mother has slapped her hard across the face the previous evening and there are finger marks on her face • Two girls have been fighting in the canteen before school • A member of staff is very upset after a disagreement with a colleague • The gym roof is leaking and you have examinations scheduled there for the day • You receive a serious complaint about a teacher’s conduct • You are due to attend a meeting in an hour with the Chair of Governors

4. What are schools about? Preventive Detention For some pupils just keeping them off the streets is a major achievement • Social function • Nobody else may take on this function Providing security and a safe environment Pupils not at ease will not learn Effective Learning Opening windows on a world Choices Realising potential Achievement and Standards Qualifications

5. Schools are complex organisations Far too complex to make decisions by hunch! Can you really know your school without detailed research and surveys?

6. Think about SELF EVALUATION What do you do already? Why do you do it? What might you do to develop your practice? Can the CEM centre help?

7. Best practice in self-evaluation A survey of schools, colleges and local authorities OFSTED REPORT JULY 2006

8. Best practice Leaders gave priority to and led self-evaluation personally. They had a clear overview of their institutions, based on an accurate understanding of strengths and weaknesses Self-evaluation was integral to the culture of the organisations. People at all levels were committed to it and fully involved. It was a continuous process, governed by the needs of the institution rather than the requirements of external bodies. It was clearly built into management systems. External inspection supported but did not replace internal review. It provided, however, the main external source of validation. An increasingly sophisticated use of a widening range of performance indicators enhanced the quality of self-evaluation. Rigorous analysis of strengths and weaknesses, particularly of teaching and learning, led to the clear identification of priorities and strategies for improvement. Sharply focused monitoring, based on clear indicators, helped institutions to measure the extent to which their work improved outcomes for pupils and young people. The views of those who received services, particularly learners but also parents and carers, were actively sought and influenced decision making.

9. Schools put the process of self evaluation at the heart of their culture • WIDERANGING PUPIL ANALYSIS • PROMOTING ACTION RESEARCH • EFFECTIVE QA SYSTEMS • PERCEPTION SURVEYS • DEVELOPING LEADERSHIP AND MANAGEMENT SKILLS • EFFECTIVELY TARGETING AND MEASURING SUCCESS • COLLABORATIVE REVIEWS • BENCHMARKING TO HELP JUDGEMENTS SCHOOL SELF EVALUATION It is better to identify a problem yourself than have others do it for you

10. 25 YEARS OF CEM CENTRE DATA Carol Fitz-Gibbon 2001 British Psychological Society It gradually dawned on me that providing the data to schools was the most important outcome of the effort, far more important than writing research papers….. The provision of data to practitioners meant that they participated in the research. Indeed they were the only ones who knew the surrounding circumstances for their classrooms, their department, each pupil, each family, etc. They were the major players: the ones who could interpret and learn from the detailed data.

11. DOING IT ALL! PRIORITISING Can school leaders find the time? Important but not urgent Is it more like? 65%-80% or 15% Urgent but not important Is it more like? 15% or 60% I keep six honest serving men Who taught me all I know Their names are What and Why and When And How and Where and Who Rudyard Kipling The Elephant’s Child

12. WHAT information do I want? WHY do I want it? WHEN do I need it? HOW do I collect it? WHERE can I find it? From WHO do I get it? The CEM centre ticks many of these boxes for schools

13. Paper based surveys: • Alis and Yellis have their ‘Extended Questionnaires’ • Yellis probes attitudes to school and student perceptions of the school and also their subjects • Alis focuses extensively on their subjects • A wealth of historical data

14. On-line attitudinal surveys • MidYIS, SOSCA, INSIGHT Yellis and Alis • Developed to meet new needs • Can be more easily adapted to meet new developments • The Parental Questionnaire • The Event mapper

15. Why use our surveys when you can write your own? • Questions re-analysed for reliability and validity • No data to process • ‘background’ of other schools enables you to see differences ,if they are significant and monitors attitudes over time

16. Some examples Yellis Attitudinal Feedback Attitudinal Comparison Graphs The Yellis Baseline questionnaire looks briefly at students' attitudes towards education and their aspirations for the near future. Feedback from this questionnaire takes the form of five graphs that look at students' job aspirations, cultural capital, likelihood of staying in education and feelings about their current school.  Basic Comparison Graphs2010.pdf .  Basic Comparison by Gender Graphs.pdf

17. Attitudinal Trend Graphs These 35 graphs illustrate how your student's attitudes change year-by-year - based on the responses given during the Yellis Baseline Test session. Year 10 Basic Comparison Trends.pdf

22. Further Attitudinal Comparison Graphs Feedback from this questionnaire takes the form of 23 graphs that illustrate how the responses of your students to the Attitudinal Questionaire differ when compared to all Yellis students in the same cohort. Year 11 Extended Comparison Graphs.pdf . 

23. Extended Comparison Graphs • The following list gives the titles of Comparison Graphs currently provided by the Extended Yellis questionnaire: • Alienation Indicators • Attitudes to Design & Technology • Attitudes to English • Attitudes to a Foreign Language • Attitudes to Mathematics • Attitudes to Science • Career-Relevant Activities Experienced • Sources of Careers Information Found Useful • Preferences for Kinds of Work • Items in the Freedom from Free Scale • Design and Technology Homework • English Homework • Foreign Language Homework • Mathematics Homework • Science Homework • Career Choice Motivators • Parental Involvement: Mothers • Parental Involvement: Fathers • Influences on Staying-On • Traumatic Events Experienced by Pupils • Places Where Pupils Feel Unsafe • Work Place Preferences • Cigarettes, Alcohol and Drugs Your data is above the Yellis average Your data is below the Yellis average Your data is about the same as the Yellis average Your data is about the same as the Yellis average

24. Further attitudinal trends These 197 graphs illustrate how your student's attitudes change year-by-year - based on the responses given on the Yellis Attitudinal Questionnaire

29. Online Attitudinal Questionnaires The Extended MidYIS Attitudinal Questionnaires will help you to answer these questions and meet the relevant requirements set out in the Framework for inspecting schools. Topics Covered Three questionnaires are included in the registration fee; the General, Induction and Parental questionnaires. General Questionnaire The General questionnaire includes a series of questions relating specifically to the completion of your Self Evaluation Form (SEF) ie bullying, racism, sectarianism, healthy lifestyles. Other sections include: Attitudes to school and study: do your students enjoy and feel happy in school? Attitudes and motivation toward subjects: what do your students feel about the classes they take? How does their motivation vary with subject? Bullying in school:do your students ever feel threatened or bullied? If so, where? Extra-curricular participation:do your students participate in additional clubs, sports or school trips? Drug use: how prevalent are cigarettes, alcohol and illegal drugs amongst your students? Career aspirations: what expectations do your students have for their future after school? Peer pressure:do your students feel pressure from their friends to behave in anti-social or anti-school ways? Socio-economic status: what are the educational levels and occupational status of the parents of your students? Cultural capital: what support for learning (books, visits to museums, libraries) is there in your students' homes? We recommend that the general questionnaire is taken in term 2 or 3. The general questionnaire takes about 45 minutes to complete. Enhance your feedback by providing your teacher groups! - If you want to enhance your subject data please provide us with your pupil teacher groups

32. Healthy Lifestyles : I take part in PE and Games lessons. I take part regularly in a sporting or outdoor activity provided by the school either at lunchtime, after school or at weekends. I take part in playground games at lunchtimes or breaktimes. I make use of the sporting facilities at school. The school encourages me to be physically healthy. The school encourages everyone to take part in sports no matter how good or how bad they are. The sporting and recreational facilities at school are excellent. I am unlikely to smoke because the school has told me about the dangers of smoking. How many cigarettes per day can someone safely smoke ? I eat less sweets and crisps now because the school has told me about the health risks involved. I eat more healthily because the school has warned me not to eat too much fried food. I am more likely to drink responsibly because the school has told me about the dangers of drinking alcohol. How many measures of alcohol (glasses of wine, half pints of beer) can someone drink before it affects their driving ? How often do you see bullying in the corridors at the change of lessons ? How often do you see bullying in the toilets ? How often do you see bullying in outdoor lessons ? How often do you see bullying at lunchtimes ? How often is there bullying via e-mail in school ? How often is there bullying by text messaging in school ? I understand the school’s policy on bullying. I know who to tell if I am bullied. The school would be able to stop me being bullied if it happened. I would tell someone in school if I saw someone else being bullied. There is a lot of bullying in school.

34. ALIS The Extended ALIS Questionnaire, generally takes about an hour to complete, covers many areas - including attitudes to school (or college), courses, Teaching and Learning and career plans. Students are also asked to comment on their responses to some open-ended questions. The questionnaire can either be taken on paper (in Term 2 of Year 2) or online (in Term 2 of Year 1).

35. How has this been used? PEDAGOGY….ALIS surveys

36. SUBJECT A 2008 Why the improvement?

37. SUBJECT A 2005 SUBJECT A 2007

38. SUBJECT A 2008 DO NOT GET TOO EXCITED!

39. We have made a comparison of perceived teaching methods as analysed by ALIS in 2004-5 with those in 2007-8. Some subject areas have appeared to change their methods radically. Others have not. Though the samples are small it is an interesting exercise to try to correlate it with the departments statistical process charts over that period. One would like to say that changes in the variety of teaching methods result in improvement but the evidence is a little tenuous so far.

40. A USEFUL STAFF DEVELOPMENT EXERCISE

45. SOSCA Attitudes to Science Survey The Science Assessment includes a survey of pupils' attitudes to science.  The feedback from the survey shows you each pupil's average score on each sub-section, on a scale of 1 to 5 from negative to positive attitudes, and their overall score.   How your school's pupils scored on each sub-section is also shown against a background of all the pupils in the SOSCA sample.  Lastly, pupils are asked to reply to three questions about science and their free-text comments are reported anonymously and in random order for your information.

46. Perception of Science This area covers all aspects of science outside the classroom and in the wider world. A typical question is: Science causes many environmental problems. Relevance of Science to Self This section explores pupils' perceptions of how science relates to them, in particular when studying science at school. A typical question is: I need to do well in science to get the job I want Enjoyment of Science This area explores pupils' enjoyment of the science they are taught in school. A typical question is: I look forward to doing experiments Environmental Empathy This section includes pertinent questions about waste and the environment to assess what the pupils have done recently with regard to actions generally considered good or bad for the environment. A typical question is: My empty cans always get recycled. .

47. Whole School Summary

50. FINALLY THE MOST IMPORTANT WORD IN EDUCATION???