PLASC/NPD User Group

1. PLASC/NPD User Group • Mick Quinlan (Mick.Quinlan@naa.org.uk) • Change, or the lack of it • The Players: suppliers and users • NC assessment process • A wander through the ‘book’… an A-Z of the collection of NC assessment data

2. Forward • I’d like to change the current process but: • there are many players in the game each with existing systems and many dependencies • we’re not good at managing change!

3. Drivers for Change • 1998 Service delivery failure of data collection exercise • 1999 Rose report – Weighing the Baby • 2004 Beasley report – service delivery failure of KS3 English test results • 2005 Statistics Commission report #23 – measuring standards in English Primary schools • 2005 Statistics Commission report #26 – school education statistics: user perspectives • 2006 NAO 3rd validation compendium report – quality of data systems underpinning PSA targets • Broadly appropriate but system needs strengthening

4. Metadata • Statistics Commission #23, Feb 2005, §11 • We feel that public presentation of the KS scores in statistical releases should include a clear statement about the uses to which the data may be put, and the limitations of the data in respect of those uses • Statistics Commission #26, June 2005, §D • Full and informative statistical commentary should be provided alongside the figures to guide users

5. Making Good Progress

6. Changes for 2006 • KS2/3 Electronic collection of TA, including pupil-level non-core results for KS3, via S2S – rather than OMR forms • A decision has been taken by the Department’s Head of Profession for Statistics not to provide this data until the quality and robustness can be assured (SFR34-2006) • KS3 English: Richard III in, Henry V out • KS3 ICT onscreen test 2nd pilot (≈170,000 pupils)

7. The Players: DfES • DfES – Paymaster/customer (TRD) • School Census/NPD and FSP/KS1 data collection • EduBase • School Self-Evaluation/Improvement • Autumn Package (benchmarks, progress, target setting) • PAT/RAISEonline) • School Accountability - Performance Tables • SERAP, FORVUS, FFT • National Statistics • Statistical First Releases • PSA Targets

8. The Players: NAA • A division of QCA (the Regulator) • Managers of the process: • Test development (NFER, Edexcel, UCLES, RM) • Pupil registration (DfES, schools) • Test Printing/Distribution (COI/Granby) • Test marking (Edexcel) • Teacher Assessments (schools, LAs) • Data capture (Pearson) • Non-standard processes (in-house)

9. The Process Schools Borderlined scripts + borderlined marksheets + school report (inc. marks/levels) Script + Revised mark and level TeacherAssessments Request forReview Request forProcess review MarkerStandardisation Jul/Aug 7.9 Jul 7.15 Oct 7.13 May-Jun 7.2 Guidance Samplescripts Mark and level Investigatereview Marked scripts Mark and level Upheldreview Dataentry MarkScripts Borderlining Oct/Nov 7.14 Jun-Aug 7.8 May/Jun 7.1 Jun 7.7 Re-review R1: Clericalcheck Samplescripts Cut scores Returnedscripts 7.12 Jul/Aug DC resultsdatabase Script pool collection Level-settingdata collection Re-markedscript R2: IndividualRe-marking May May/Jun 7.3 7.5 Selected scripts 7.10 Jul/Aug Malpractice Scriptscrutiny Level- setting R3/GR: Group review May/Jul 7.14 Recom-mend-ation Jun 7.4 Special consideration Jun 7.6 7.11 Jul/Sep Pretest 2: Draft Level thresholds May/Jun 7.13

10. A is for Assessment • National Curriculum • key stages: Foundation, KS1, KS2, KS3 • Subjects • core: English, maths & science • Attainment Targets • non-core: Art & Design, D&T, Geography, History, ICT, MFL, Music and PE • Test/Task & Teacher Assessment • Levels

11. Levels • Q: What’s the difference between Level 4 and Level 5? • A: 1 mark • Making Good Progress (KS2-KS3,page 9) • This shows that 15[%] pupils did not progress by a full level since finishing KS2 and three[%] have actually regressed

12. A • A = pupil Absent from the test • Aegrotat • Achievement & Attainment Tables • Age Standardised Scores (ASS) • Additional time • Appeals (process reviews)

13. BCD • B = Working Below the level of the test • Borderlining • Braille • Compensatory award • Classes/Teaching Groups – collected in 1997 • D = pupil Disapplied from the curriculum • Date of Birth

14. E • E = Exceptional Performance • Extension Papers • Early takers • Errors

15. FGH • F = Future sitting (subjects early takers not taking) • Fine grade • Foot and mouth • Gender • Handwriting

16. IJKL • Independent schools • L = pupil Left school (KS2/3) • L = below Level 2 in the readingcomprehension test (KS1)

17. MNO • M = Missing scripts • Maladministration/malpractice • Missing pages • Mental Maths test • Modified large print (MLP) • N = No test level (too few marks gained) • Names, of pupils • Overseas/Offshore schools • Optional tests (Y3-5 & Y7/8)

18. PQR • P = Past sitting (subjects early takers took last year) • Progress tests (Year 7) • P Scales • Q = result annulled - malpractice (DfES) • Reading • Reviews • Re-sits

19. ST • Service Children’s Education (SCE) Schools • Special Consideration • Shakespeare • Sub-Levels • Spelling test • T = Pupil unable to access the tests • Tier

20. UVW • U = No TA subject Level calculated (KS1) • Unique Pupil Number (UPN) • V = mixed tiers • Versions • W = Working towards Level 1 • Writing

21. XYZ • X = Marks pending (KS2/3) • X = pupil not entered for readingcomprehension test (KS1) • Year groups • Z = ineligible pupil