LAT Value Added Knowledge Share Part 2

1. LAT Value AddedKnowledge SharePart 2 YPLA Strategic Analysis & Research team Championing Young People’s Learning

2. The Methodology Championing Young People’s Learning

3. Value Added Specification It’s long and it’s got some complex equations Producing a version with a bit more plain English in it Methodology independently reviewed by NfER Underlying concepts are complex… …but individual steps are not so bad Championing Young People’s Learning

4. Multi Level Modelling: More detail MLM applied at qualification & subject level (e.g. A level History) Attempting to fit a polynomial equation Complexity of equation based on number of records & insts Where there are very few records things are grouped together at SSA level Championing Young People’s Learning

5. Multi Level Modelling: Polynomials Quadratic (order 2) A + Bx + Cx2 80 to 500 cases 500 to 5000 cases 5000+ cases Cubic (order 3) A + Bx + Cx2 + Dx3 Quartic (order 4) A + Bx + Cx2 + Dx3 + Ex4 Championing Young People’s Learning

6. Multi Level Modelling: Polynomials Quadratic (order 2) A + Bx + Cx2 80 to 500 cases 500 to 5000 cases 5000+ cases Cubic (order 3) A + Bx + Cx2 + Dx3 Quartic (order 4) A + Bx + Cx2 + Dx3 + Ex4 A, B, C, D and E are known as ‘gamma’ coefficients Championing Young People’s Learning

7. Multi Level Modelling: Provider Lines Championing Young People’s Learning

8. Multi Level Modelling: Provider Lines • National Line = 10 – 5x + 0.5x2 + 0.01x3 • Provider A Line = 12 + 10x + 0.5x2 + 0.01x3 • Provider B Line = 1000 – 3x + 0.5x2 + 0.01x3 • Provider C Line = -400 + 10x + 0.5x2 + 0.01x3 Championing Young People’s Learning

9. Multi Level Modelling: Equations National Line Provider Line Variation Distribution Of Results (Variance) Error Championing Young People’s Learning

10. Multi Level Modelling (S+ and R) Cannot be done with SQL or version of SPSS we have R & S+ are programmes that support MLM calculation Data Service use S+ but has licence issues YPLA getting open source programme R onto estate Both use the same programming language Code used works in a similar way to SPSS Championing Young People’s Learning

11. Multi Level Modelling Basic R Syntax base<-read.table('C:\\DriveD\\LAT VA\\Q_111_S12330.dat', header = TRUE, fill = TRUE) library (lme4) a<-lmer(POINTS~PRIORC+PRIOR2C+PRIOR3C+PRIOR4C +(PRIORC+PRIOR2C|LAESTAB),data=base) fixef(a) Championing Young People’s Learning

12. The Calculation Process Championing Young People’s Learning

13. Step 1: Data from FFT Received in SPSS format FFT can advise on issues with the data Includes data fields on: Provider code (UPIN & LAESTAB) Qualification type (A09 and LAT VA qual codes) Prior attainment and outcome attainment Learner Names Championing Young People’s Learning

14. Step 2: Check and sort data Data service do a range of checks on the data Drop any qualifications that are too small to include Check all providers we expect to be included are in the data Produce “centred variables” They then split into lots of small files (one per qual & subject) Championing Young People’s Learning

15. Centred Variables Many equations include quartic term (i.e. x to the power 4) For a prior attainment score of 58 this is a big number 58 x 58 x 58 x 58 = 11,316,496 To make the MLM calculation run quicker the variables are centred This means smaller numbers are used Involves some fiddly calculations but just basic maths If you hear reference to “Prior C”, “catalyst file” or “beta variables” these are interim steps used in this centring process Championing Young People’s Learning

16. Step 3: Apply MLM Data is fed into S+ programme one file after another This will fit the national line and give details on distribution of results Gives an output as text file These text files are then grouped together using a compiler routine Championing Young People’s Learning

17. Step 4: Check solutions The individual solutions are fed into a spreadsheet to check whether they look reasonable This spreadsheet is known as the “Batch LAT” Original FfE version was very complex (Over 6000 lines of code) For 2009/10 we will be using a simplified version (1000 lines of code) General checking by eye Some mathematical checks too (positive definite matrix) Produces “decentered coefficients” Championing Young People’s Learning

18. Step 5: Re-apply MLM If the solution does work then apply a lower order equation …or group data up to SSA level Then re-check solution End point of this is national lines for all of the qualifications Championing Young People’s Learning

19. Step 6: Upload data to Online LAT The national lines and individual provider data are uploaded into the Online LAT This undertakes the same calculations as the Batch LAT It also generates reports for all providers These are viewable through the provider gateway Championing Young People’s Learning

20. Step 7: Extract result files Bulk data files can be exported from the Online LAT The “new style” reports will be generated using these files Results files sent to OfSted to be used in their reports Results files uploaded to SQL for use by YPLA Results for 2007/08 and 2008/09 are on MISVS001 in LSC_MI_DB_PUB (the filenames start LATVA_) Championing Young People’s Learning

21. The ReadyReckoner Championing Young People’s Learning

22. Ready Reckoner • Providers were keen to get an early view of LAT VA scores • Excel spreadsheet that allows them to model their own data • Due for release in early October • Uses data from LAT VA 08/09 amended data release • First line support by Data Service’s Service Desk • Second line support by YPLA

23. Input Data

24. Output Data

25. Work scheduled for the next 6 months Championing Young People’s Learning

26. Timescales • Unamended run mid November • Updated LAT Handbook and communications at same time • Amended run planned for mid January

27. Conclusion Championing Young People’s Learning

28. Remember the basics A B A level result C D National average achievement E E D C B A Average GCSE grade