Severe Grading in MFL - the people

1. draft Sat 18th Severe Grading in MFL- the people ALL = Assoc. for Language Learning Helen Myers (Past President, ALL & Asst Head, The Ashcombe School) David Blow (Head, The Ashcombe School) Duncan Byrne (Chair ISMLA) Geoffrey Plow (Exams, ISMLA) John Dunford (Gen. Secretary ASCL) Peter Downes (former Chief ML examiner, Head, President ALL & SHA) ISMLA = Independent Schools ML Assoc

2. Severe Grading in MFL- brief history Long-standing concern - research by Nuttall in 1974; Peter Downes raised over time, leading to recommendation in the Nuffield Report Series of presentations by Helen Myers and David Blow using DfES and other data Continuing concern in ML community, esp as Languages is no longer compulsory and numbers go into freefall Series of meetings by ALL, ISMLA etc with QCA and exam boards (‘06)

3. GCSE MFL grading • Our outline principle is that students studying a particular GCSE subject should have a reasonable expectation that they will get comparable grades across a range of subjects. • The PANDA Relative Performance Indicator measures the performance of students in their other subjects compared with the particular one. By the principle above, these should be negligible. They are not.

4. thanks to Peter Tymms for this phrase “Severe grading” Terminology is important. Words like “hard”, “standards”, “demand” carry multiple connotations. Restate principle that pupils of comparable prior attainment should attain comparable grades across a range of subjects whether Business Studies which will be only started at GCSE, Maths, Drama, etc all of which have very varying situations for different pupils

5. Depressing exams (NB separate issue from grading) As professionals, we know (or should know) that a “difficult” paper with low scores does not need to lead to low grades as the grade boundaries /conversion to UMS will be adjusted to compensate... BUT pupils leaving that exam will perceive that they have performed badly, which compounds the “severe grading” problem

6. Issues for pupils • Perceive that they are performing worse in languages in other subjects • Reduces the take-up at KS4 and A-level • In each case, pupils are doing relative comparisons on grades (note key issue at AS in French as compare with other subjs - at A2 Univs can make subj. by subj. adjustments)

7. Issues for staff & managers • Heads and Governors may think that languages are “under-performing” • Mixed messages re school targets and DCFS targets • Ritual debate each year about whether standards are slipping / pupils are choosing “soft” subjects........

8. Dearing Review National outcry at drop in numbers GCSE Aug 06 - Alan Johnson sets up Dearing Review “Mr Johnson wants to see what more can be done to encourage14-16 year olds to study GCSE or other language courses leading to a recognised qualification.” DfES Press Notice 2006/0144 Oct 06 Meeting Nov 06 with Lord Dearing Idea of broad comparability with Maths grading Awareness of public and political sensitivity

9. Numbers - GCSE French ‘02 German - similar pattern but 40% as many, e.g 120,000 (French 300,000) ‘06

10. Grades ‘02 ‘08 drop of 30,000 in number gaining A*-C from ‘02 to ‘06 drop of 40,000 in number gaining A*-C from ‘02 to ‘08 German - similar pattern but 10,000 (‘02 to ‘06) and 18,000 (‘02 to ‘08)

11. Joint Proposal to Dearing (1) The mandate of QCA and the examination boards is to ensure that "standards are maintained over time". This is an enormously challenging task and one which has a high public profile. Comparability of grading It is now accepted by QCA and the exam boards that pupils of similar prior/concurrent attainment will gain a lower grade in languages than most other subjects. The presented documentation is available at http://www.all-london.org.uk/severe_grading.htm Because this difference is embedded historically, and their mandate is to maintain standards over time, a decision from the DfES is needed to change this.

12. Joint Proposal (2) Opportunity for change In the light of the crisis at KS4, there is now the opportunity for a change. The very strong experience of modern language teachers, advisers and senior managers is that the discrepancy in grading is one significant negative factor in the decline. Making changes here will not in itself solve the overall problem, but, to use a metaphor, will remove having a hand tied behind one's back.

13. Joint Proposal Proposal That QCA, and thus the exam boards are mandated to ensure that the grades awarded in Modern Languages move to become comparable, on average, to those awarded in Mathematics. Different methodologies exist to measure the comparability (described in the papers), and these should all be used, so that the pupils of similar prior/concurrent attainment have a similar average grade in Modern Languages to that in Mathematics. This proposal should be treated independently of the many other ideas and proposals to deal with the issue of KS4 M.L.

14. Dearing Review Led to recommendation in Consultation Report (Jan 07) confirmed in Final Report (Mar 07) to have definitive study published on “perception” of severe grading (with implicit corollary that there should be action...) - the consultation had “found strong confirmation of the view that award of grades is more demanding than for most other subjects”

15. Dearing Review - press Dr Coe, of Durham's curriculum, evaluation and management centre, believed such trends were repeated in other years, but insisted "the question of difficulty is not about content of the subject ... It is purely about the examination and grading process." David Willetts, Conservative education spokesman, said: "If there is evidence modern languages is tougher than other GCSEs, then that is something that has to be corrected. They should be the same level of challenge as traditional academic GCSEs."http://education.guardian.co.uk/gcses/story/0,,2031769,00.html James Meikle, education correspondent Monday March 12, 2007

16. Political context “ I would like to wait until I have this advice before making any decisions about changing grade boundaries” Alan Johnson, Secretary of State, May 08 referring to the QCA study set up in response to Dearing recommendation when replying to joint letter from ASCL, ALL, ISMLA Meetings with Jim Knights, David Laws (LibDem) and Nick Gibb (Con) to ensure cross-party awareness

17. Sensitivity We do not underestimate the sensitivity of this topic Remember Feb 08 “get GCSE ML without speaking” “ML Orals abolished!” Through all of our involvement, have sought to make politically realistic proposals - e.g Lord Dearing liking the idea of comparability with Maths “no-one will say that Maths is easy”

18. QCA report - Feb 08 Introduced two definitions of comparability: 1. “Attainment–based comparability” - § 8 “each boundary must be set using professional judgement. The judgement must reflect the quality of candidates’ work, informed by relevant technical and statistical evidence.' - § 8 “Fundamentally, awarding meetings are required to ensure that the quality of work describing a particular grade this year is the same as that of the previous year.” - § 9

19. QCA report - Feb 08 2. Aptitude–based comparability The ‘aptitude-based’ conception of comparability holds that two examinations may be seen as comparable if students of a certain ‘calibre’ have an equal chance of achieving a particular grade in any examination (either within subject areas or between them). One examination is harder than another if the results of a (representative) group of students taking both examinations are worse in it. This is the approach to comparability on which the ALL proposal is based.

20. statistical case of “severe grading” now accepted - see joint press release QCA report - Feb 08 What if French grades based on candidates’ key stage 3 test scores and their results in GCSE mathematics? The analyses indicated that the changes would be marked at the higher grades. About half the candidates presently awarded a grade B would gain a grade A as the threshold mark or performance standard for a grade A would have to move down by about half a grade width. There would be a similar effect at grade C. evidence suggests that for D grade in French 70% gain higher grade in Maths QCA report had said that only 123 schools (excluding grammar schools) had maintained a high rate of GCSE languages take-up (over 90 per cent entered in each year from 2003 to2006).

21. QCA report - Feb 08 Conclusion: “The present examination system is not based on an aptitude-based conception of comparability and its adoption would create a major threat to public confidence in students’ results. We have similar concerns about applying a new performance standard to languages but not to any other subjects, some of which might also want to claim special status. We do not have evidence that there have been significant changes to grade standards in GCSE languages in recent years. In conclusion, we do not therefore, recommend any adjustment to national grade standards in GCSE languages.” unfortunate implication in conclusion of report that T&L special problem with MFL :“Instead we should focus on improving levels of teaching and learning in modern languages in order to gain students' commitment and raise performance.”

22. Brigshaw School & Language College presented at Language College Conference May 08 and Oct 08 2007 GCSE results: 53% 5A*-C; 40% 5A*-C inc En + Ma 24% A*-C in MFL i.e. 59 out of 243 comparing MFL results with Maths: 70% lower grade than Maths 23% same grade in Maths 7% higher grade than Maths so in Year 11 those not in line for C switched to Asset Languages (138 of 240) of which expect 46 to get equiv grade C or higher. So (46 +102) / 240 = 60% A*-C in MFL for Jun ‘08 but is it T&L issue?

23. Brigshaw School & Language College “What is most striking about the school is the pervasive sense that it is a place where learning languages matters. The school has extensive provision for languages, with good links across subjects so that pupils can see how and why they can apply their language skills and knowledge. Pupils appreciate the very good opportunities to get involved in visits abroad. New courses are ensuring that, whilst everyone has to study a language throughout their time at school, there are courses to suit different needs. Extensive use is made of the expertise in the language college to support other schools and the community.” Ofsted Nov 06 The school had also called in the examiners to look at the papers, and they agreed that the school had properly prepared the candidates

24. Way forward We believe it is unrealistic to move to comparable grading for all subjects Currently 4 main bands ML Maths, Sci, His, Geo English Art, Drama, PE, We are proposing simply that ML moves into the 3rd band - tiny changes in grade boundaries involved, similar to those which have taken place - “one-off step change” data from CEM SCORE paper

25. Situation of MFL - A/L To asses the relative performance of subjects in the context of uneven grading at a national level, there is a calculation in the PANDA of 1) the average pts score of candidates in the subject 2) the average points score of those candidates in the OTHER subjects they are taken. The difference between the two figures shows how the grading for the subject compares to other subjects. Because the second figure is calculated ONLY for the candidates doing the first subject, it automatically takes account of the ability profile of the candidates doing the subject This establishes the national context, and if the same calculation is done for the school, it removes the "severe grading" issue from the consideration of performance nearly 1 grade on average from post-16 PANDA national data for Relative Performance Indicator (RPI) 20 points = 1 grade

26. Situation of MFL - A/L French - blue German - black nearly 1 grade on average Col. 3 "difference" as in previous graph from post-16 PANDA national data for Relative Performance Indicator (RPI) 20 points = 1 grade

27. Situation of MFL - A/L French - blue German - black Col. 3 "difference" as in previous graph C B 0 89.0 -7.2 Col. 1 - average score in subject 20 points = 1 grade

28. Situation of MFL - A/L French - blue German - black Col. 3 "difference" as in previous graph C 7.2 B 0 96.2 -7.2 20 points = 1 grade Col. 2 - average score in OTHER subjects (related to ability profile)

29. Issues for MFL - A/L • Note that French & German have high “severe grading factor” at A/L. • French & German have the highest average grades in OTHER subjects • So • will suffer in comparison with other subjects, • are only left with the most able • Compounded by situation at AS: for these 2004 A/L grades,in 2003 at AS French had: subject ave 22.9OTHER subject ave 29.9 (diff = - 7.7)

30. A virtuous circle??? • Language grading is fair, so... • Pupils choose MFL at KS4, feel good after their exams and get grades in line with other subjects so... • More choose at AS and get grades in line with other subjects so... • More continue to A2 and so... • Everyone is happy!