Overview

1. Overview • Contextual VA • What is it? • What factors are used? • How is it calculated? • Self-Evaluation Analyses • Overview • Examples (NB most examples shown are KS2, but same principles apply at KS3 and 4) • Slide 21 example applies to secondary schools only • Interpretation • Limitations • Extensions

2. Contextual Value-Added • Contextual Value-Added • compares • the progress made by each pupil • with • the average progress made by ‘similar pupils’ in ‘similar schools’ • calculates • the average progress made by all pupils in the school • converts • the school’s value-added score into a PERCENTILE RANK

3. Factors used in new Contextual VA models: • Pupil Factors • Overall attainment in previous Key Stage • Variation between subjects at previous Key Stage • Variation between Test and Teacher Assessment • Gender • Month of Birth • School Factors • Demographic Data (linked to pupil PostCode) • FSM Entitlement • Average prior attainment for all pupils • Distribution (spread) of pupils’ prior attainment • Factors used in the ‘SX’ model include: • Mobility • Ethnicity • SEN Stage • EAL

4. Value Added Models - Overview • 3 VA models are used in Self-Evaluation Reports: • PA and SX models are used for value-added sections • PA and SE models are used for estimates • Model PA (Prior Attainment) Prior Attainment Gender Month of Birth • Model SE (Socio Economic) Prior Attainment Gender Month of Birth School Context • Model SX (School EXtended) Prior Attainment Gender Month of Birth Pupil Context School Context (E)

5. VA Models - Indicators • Nonlinear modelling and some multilevel modelling used to improve accuracy across whole attainment range

6. Analyses – Outline

7. Contextual VA (KS12) • Includes: • Value-added analysis which takes into account: • Pupils’ prior-attainment • Pupil and School Context factors. • Summary of 3 indicators for each Subject: • e.g. (KS2) %Level 4+, %Level 5+, Overall Level • Percentile rank • Indicators for pupil groups • Girls, Boys • Lower, Middle and Upper Prior Attainment • Highlights • Where VA is significant • Where floor target not met (e.g. KS2, L4+ below 65%) • Where match to prior attainment less than 85% • Enables • Initial overview of trends for 3 years • Evaluation of variation within and between core subjects

8. Contextual VA (KS2 Core) Percentile rank of the difference between actual result and estimate, compared with all schools nationally. 1 = value-added is in the top 1% of schools, 100 = VA in the bottom 1% of schools. Middle 50% of schools are in the range 26 to 75. Number of pupils in each of the 6 groups: Boys and Girls split by prior attainment into Lower, Middle and Upper (highlighting showing significant difference between actual performance and expected)

9. Value Added (3 Year Summary) • Includes: • Data for 3 years overall and each individual year • School Context measures • Unadjusted (Raw) results • Value Added (using prior-attainment only) • Value-Added (using prior-attainment and contextual factors) • Enables: • Indication of impact of school context, by comparing VA with and without contextual factors • Assessment of overall trends for broad indicators covering the Key Stage as a whole

10. Overall Summary (Part 1) These percentile ranks show school’s raw results compared with national. No highlighting used here.

11. Overall Summary (Part 2) • These percentile ranks are based upon actual results compared to what would have been achieved if: • pupils had made similar progress to similar pupils nationally • pupils had made similar progress to similar pupils in similar schools nationally.

12. Some combinations of arrows and boxes Yr1 Yr2 Yr3 Trend Consistently significantly above Significantly above, but declining Significantly above, but declining rapidly In line, but declining In line and no trend Consistently significantly below Significantly below, and declining Significantly below, and declining rapidly In line, and no trend due to variation (down then up) Significantly above and no trend In line, and improving rapidly

13. Value Added (Significant Areas - Grid) • Includes: • Only areas where there is a significant difference • Where overall VA across 3 years is significantly high or low • Where the trend over the 3 individual years is significant • Where there are 10 or more pupils in the group • A range of groupings including by gender, prior-attainment, ethnicity, SEN stage and FSM entitlement. • Enables: • A more detailed evaluation of significant strengths and weaknesses and of 3 year trends

14. Significant Areas (Grid) These boxes refer to the 3-year trend. See slide 12 for more information.

15. Estimates Summary • Includes: • Estimates for attainment in future years • Based upon 3 VA models: • Prior Attainment (PA) • Prior Attainment + School Context (SE) • Top Quartile (of schools in similar contexts) • Also includes estimates which takes into account the school’s previous VA performance (averaged over 3 years) • Enables: • A discussion about estimates which: • is based upon consideration of a range • Indicates the likely outcomes if previous school performance is maintained

16. Estimates (L4+) 3 types of estimate (PA, SE and TQ) shown in ascending order. Estimate in box based on your school’s historical performance.

17. Estimates (L5+) 3 types of estimate (PA, SE and TQ) shown in ascending order. Estimate in box based on your school’s historical performance.

18. Value Added (Significant Areas - Text) • Includes: • Only areas where there is a significant difference • Where overall VA across 3 years is significantly high or low • Where the trend over the 3 individual years is significant • Where there are 10 or more pupils in a group • A range of groupings including by gender, prior-attainment, ethnicity, SEN stage and FSM entitlement. • Looks at a limited subset of indicators (when compared to the ‘grid’ approach) • Enables: • A more detailed evaluation of significant strengths and weaknesses and of 3 year trends

19. Significant Areas (Text) This is the same information as shown in slide 14, but in text version.

20. Secondary Schools Only • The following report (KS234 CVA) shows value added from KS2-3, KS3-4 and KS2-4. • Each row relates to one particular cohort.

21. KS234 CVA In this example, performance at KS3 is significantly lower than could be expected. Performance from KS3 – KS4 is significantly better than expected. Overall, from KS2 – KS4, performance is significantly better than expected.

22. Limitations of the Analyses • VA based only upon pupils matched • So need to be aware of this in schools with significant proportion of refugees or pupils for whom previous data is not available • Significant differences • Not shown where number of pupils falls below 10

23. Extensions • The school may have access to more detailed reports, for example: • Analyses provided by FFT to LEAs which look at VA and estimates in greater detail, including pupil-level analyses. • Outcomes of analysis using PAT (Pupil Achievement Tracker) to examine performance of particular groups of pupils or specific aspects of the curriculum (using question-level analysis). • Other Value-Added Analysis (e.g. using MidYis, Yellis, CATS) – in such cases, a ‘triangulation’ approach might be helpful.

24. Triangulation of Data Analysis B e.g. PAT Analysis A e.g. FFT Professional Judgement Basis for action Investigate Further Challenge Assumptions Check Accuracy