LAT Value Added Knowledge Share Part 1

1. LAT Value AddedKnowledge SharePart 1 YPLA Strategic Analysis & Research team Championing Young People’s Learning

2. Topics to be covered What is LAT Value Added? The history of Value Added systems What a provider would need to know LAT VA methodology LAT VA calculation process Ready Reckoner Data Dashboard

3. What is Value Added? Championing Young People’s Learning

4. JACK JILL Mostly at grade C GCSEs Mostly at grade A A-level Biology Result Grade B Grade A Which one of Jack & Jill “did better”? Championing Young People’s Learning

5. Comparing Using Prior Attainment A B A level result C D National average achievement E E D C B A Average GCSE grade

6. What does that look like in numbers? Both Jack & Jill did better than expected… …but Jack had more “value added” in terms of progression from Level 2 to Level 3

7. Concept of Value Added: Performance at level 3 (key stage 5) is linked to performance at level 2 (key stage 4) Providers who recruit only those with good GCSEs will get better A level results – but are they better providers? Value added tells us how well providers do in comparison to others who have learners with similar prior attainment Championing Young People’s Learning

8. A History OfValue Added Championing Young People’s Learning

9. History of LAT Value Added Originally developed by Framework for Excellence in 2005 Underwent extensive piloting, testing and review Published on the Provider Gateway Used by OfSted in inspection process DCSF then developed own measure (“CVA”) to give provider level value added scores Ownership transferred from FfE to YPLA & Data Service Championing Young People’s Learning

10. Comparison with – Success Rates • LAT Value Added • Achievement only • Level 3 graded quals only • Looks at grade achieved • VA scores are benchmarked against national performance. • Success Rates • Includes retention • All levels & qual types • Only looks at pass or fail • Result is a percentage which can be compared to national averages

11. Comparison with – DfE Value Added • LAT Value Added • Qual & subject scores only • Only uses average prior attainment • Average score is 0 • DfE Post 16 Key Stage 5 VA • Institution level score only • Uses 20+ factors • Average score is 1000

12. Comparison with – ALPS & ALIS • ALPS & ALIS • Produced by third parties • Uses partial data set • Use other statistical approaches • Used for internal quality improvement • LAT Value Added • Produced by government agencies • Uses national data set • Uses multi level modelling • Used by OfSted

13. What a provider would want to know about Value Added Championing Young People’s Learning

14. Calculating a Value Added score

15. How is Prior Attainment calculated? • Data sourced by Fischer Family Trust for YPLA • Average of all aims at level 2 and below taken in the last 2 years • Means a learner with 3 GCSE A grades has a higher score than a learner with 8 A grades and 2 B grades. • Different aims have different weightings (GCSE = 1, GNVQ = 4) • Not all aims have the same maximum score • Prior attainment calculator on provider gateway

16. Prior Attainment – QCA Points

17. Prior Attainment – Examples • A Learner attains • Six GCSEs at grade C at age 15 • A merit in an intermediate GNVQ at age 16. • The learner then progresses to take an A-level at age 18. • Prior attainment for this learner is all attainment up to, and including, age 16: • all six GCSEs are assigned 40 points • an Intermediate GNVQ merit is assigned 184 points (equivalent of four GCSEs) • Total prior attainment is divided by the total size of all qualifications attempted: • ((six GCSEs x 40 QCA points) + (184 for the GNVQ)) • (six GCSEs + the equivalent of four GCSEs) • = 424 ÷ 10 = 42.4 QCA points.

18. How is Outcome Attainment calculated? • Sourced by Fischer Family Trust from awarding bodies • Used by DfE in Achievement & Attainment Tables • Providers have a chance in the Autumn to correct the data • https://tableschecking.education.gov.uk/ • Hence un-amended (November) and amended (January) data • AS levels that are not cashed in do not appear in the VA data

19. Rebased QCA points • The actual and expected attainment is shown in QCA points • Re-based to get rid of cliff edge with fails

20. Rebased QCA points • The gaps between grades differ for different qualifications • i.e. 30 points between A level grades • but 15 points between AS grades • Means that a score of +30 for A levels means the same thing as a score of +15 for AS levels

21. Rebased QCA points

22. Multi Level Modelling • So how is the “expected attainment” line calculated? • Uses statistical technique called “multi level modelling” • Basically an advanced method of getting a line of best fit… • …like when you right click on a chart in Excel to get a trendline • Multi level modelling used because it analyses the performance of each individual provider AND national picture • Used in education research for the last 20 years

23. The standard “line of best fit” approach

24. Multi Level Modelling approach

25. Calculating a Value Added score

26. Shrinkage Factor • A number between zero and one • For large cohorts, shrinkage factor ~ 1 (no impact on result) • For small cohorts, shrinkage factor much less than 1 - reduces the score towards the national average • Makes results for small cohorts more reliable and less volatile • Large groups or subjects remain virtually unchanged. Few Cases Many Cases Smaller Shrinkage Factor 1

27. Confidence intervals • Value added results are shown with a 95% confidence interval

28. Confidence intervals Confidence intervals are wider for small groups of learners and where results are inconsistent

29. Aggregation • Initial results are at qualification and subject level for each provider (e.g. A level Biology) • These are aggregated up to qualification level (e.g A level) • …and to sector subject areas (e.g. Science & Mathematics) • There is no overall value added score for a provider as combining results from different qualifications does not give a reliable result • Results files include data at qualification and SSA level

30. What gets published? Championing Young People’s Learning

31. Where to find reports & guidance • Providers access their data through the Provider Gateway • Previous years guidance on SFA website but YPLA are preparing guidance for 2009/10 data release • 2009/10 data due for initial release in mid November • YPLA LAT VA webpage: • http://www.ypla.gov.uk/aboutus/ourwork/qa/lat-va-tool/

32. New report layout For the 2009/10 release a set of pdf reports are being developed to complement the information on the provider gateway New reports are simpler to understand

33. New report layout Colour coding Green = significantly above average Blue = significantly below average Multiple years data Breakdown by age Only covers qualification scores – not national comparison charts

34. Provider Gateway Reports Reports on the provider gateway same as previous reports Very powerful statistical tools… …but difficult for someone new to VA to understand

35. Summary Report

36. Above Average: Statistically Significant Above Average: Not Significant Below Average: Statistically Significant Below Average: Not Significant

37. National Comparison Chart National Line (Solid) Provider Line (Dotted)

38. Above average at all points Below average at all points Below average for low PA Above average for high PA

39. Chances Chart

40. Final points • Where providers are part of a consortia there data is merged together and shown for lead provider • There was a “Distance Travelled” measure that looked at ungraded qualifications – not calculated since 2006/07