Implementing measures of success: training resource

1. Implementing measures of success: training resource This presentation is designed to assist your organisation in its staff development and training regarding the measures of success. It contains speaker notes which can be viewed or printed in notes page view It can be downloaded and delivered as it stands, or can be customised to meet your needs It is intended for use in presenting the measures to: Curriculum Managers, Quality Managers and Course Leaders in colleges The presentation, including notes, can be downloaded from the LSC website: www.lsc.gov.uk/ffe

2. Section 1Introduction

3. What are the measures of success? • They are a suite of measures developed to measure the success of learners and the performance of providers more accurately and fairly, thus helping to demonstrate the sector’s existing strengths and to identify areas for improvement. • There are currently seven measures being developed for the learning and skills sector. These are: • Value added (VA) and distance travelled (DT) • Qualification success rates • Recognising and recording progress and achievement (RARPA) • Learner satisfaction • Learner destination • Responsiveness to employers • Staff measures

4. Which of the measures apply to my organisation? The measures which are currently applicable to colleges are: Qualification success rates VA and DT RARPA These and the other measures are being developed as part of the LSC’s Framework for Excellence.

5. How are the measures used? The measures: integrate key information used for quality management with that used by the inspectorates and the LSC provide you with powerful tools for the analysis of performance at organisation and subject/qualification level can help you monitor the progress of different groups of learners enable you to highlight strengths and weaknesses in performance relating to learners’ progress and achievement stimulate you to review quality management processes and target quality improvement strategies enable you to help individual learners maximise progress and achievement are to be used for quality management and accountability

6. What will the measures enable me to identify? • Does the area of learning for which you have responsibility: • exceed / meet / fall short of qualification success rate benchmarks? • perform better / as expected / worse for VA and DT? • meet / fall short of the standards for RARPA? • In all cases: what actions are you taking to remedy identified weaknesses and to celebrate and build on strengths?

7. Inspectorates’ use of measures of success • The Inspectorates have been piloting and are gradually introducing the measures • They will be encouraging colleges to ensure that: • You have accessed measures data and information • You have used the information to help evaluate the quality of provision • You have planned or taken action as a result of reflecting on the data and information

8. Section 2Relationship of the measures to the LSC’s Framework for Excellence

9. Framework for Excellence • The Framework for Excellence is a new, all-inclusive and radical approach to managing performance across the learning and skills sector. • Further development of the measures is being taken forward through the Framework for Excellence • All of the measures of success will be included in the evidence for the seven key performance indicators that form the basis of the Framework for Excellence.

10. Overall rating Finance Responsiveness Effectiveness Quality of outcomes Financial health Delivery against plan Quality of provision Financial control Responsiveness to employers Responsiveness to learners Framework for Excellence: Key Performance Indicators Measures data

11. Section 3Qualification success rates

12. What are qualification success rates? Similar to data already used by your college Benchmarks adjusted by level and length of qualification Apply to all learners on LSC-funded courses Tables for all learners and also broken down by 16-18 years and 19 and over Provide the means for you to make valid comparison at headline level between your college’s performance and that of similar colleges

13. Length and level adjusted benchmark table

14. What information do qualification success rates provide? Show your overall performance by broad qualification type Allow you to look at performance in specific qualification / subject areas Allow you to identify areas of strength and weakness For 16-19 learners you should use them in conjunction with VA and DT data

15. How should I make use of qualification success rates data? To investigate how well your area of learning compares to national benchmarks. What areas give you cause for concern? What areas should you celebrate and build upon? In combination with success rate data from other sources, such as the LSC Benchmarking website and College Performance Reports, to help monitor the performance of different groups of learners, by gender, ethnicity, age and learning difficulty and/or disability In conjunction with VA and DT data to arrive at a rounded picture of performance Most importantly, you need to identify and act on areas of provision that are at or below minimum performance benchmarks

16. Section 4Value added and distance travelled

17. What are value added and distance travelled? VA and DT: • are powerful tools for the analysis of performance at college and subject/qualification level. • aim to show performance at subject-qualification level, based on the outcome of individual students taking into account their prior attainment • are available to colleges via the LSC’s Learner Achievement Tracker (LAT), which is a free on-line analysis tool

18. What do VA and DT cover? VA covers Level 3 qualifications with graded outcomes such as A levels and BTEC diplomas and certificates. DT covers Level 3 qualifications that don’t have graded outcomes and also covers the majority of the larger qualifications at levels 1 and 2. These include GCSE, BTEC First and intermediate NVQs, etc.

19. How are VA and DT relevant to me and my college? VA and DT: provide you with a powerful tool for the analysis of performance at college and subject/qualification level help monitor the progress of your learners enable teachers to set targets for learners and assess performance of specific groups by subject-qualifications enable you to highlight strengths and weaknesses in performance relating to learners’ progress and achievement enable your staff to assist individual learners to maximise progress and achievement provide evidence that will be used by Ofsted and local authorities

20. What will VA and DT information help me to address? VA and DT information will prompt you to consider: Whether the area of learning for which you have responsibility performs better/as expected/worse for VA and DT What actions you will take to remedy identified weaknesses and to celebrate and build on strengths

21. How are Ofsted using VA and DT in colleges? • Ofsted have been piloting and are gradually introducing the measures • You are encouraged to ensure that your staff: • access VA and DT data and information • use the information to help evaluate the quality of provision • become involved in action taken or planned as a result of reflecting on the data and information

22. What do VA and DT tell me? VA and DT will: • show colleges’ strengths and weaknesses by subject, qualification and Subject Sector Area (SSA) • help set targets for learners and assess performance of specific groups by subject-qualifications, through the LAT. • Illustrate details of the performance of individual learners

23. VA and DT in the Learner Achievement Tracker (LAT) There are three types of output for both VA and DT in the LAT: • Summary reports by qualification and SSA • National comparison graphs and • Chances charts There is also the facility for ad hoc national comparison graphs and chances charts reports by subject-qualification for VA

24. VA summary charts Your college’s overall VA summary chart could help answer the following questions: • In which areas are we performing well or badly? • What is performance like in our core areas ? • Which areas does the college need to explore further in terms of performance? • Are there any surprises in the summary chart? • Does our self-assessment/college development report reflect the performance shown in the summary charts? • What do we know about quality assurance and student support mechanisms in the areas which have performed well or badly?

25. VA national comparisons graph and ad hoc reports Using the LAT, providers can estimate VA scores for smaller groups within subject qualification using the ad hoc reports. Ad hoc reports: • allow you to identify performance by cohort accurately without engaging in the actual calculation • can be saved in the LAT for up to three months to be reviewed by more than one user in the provider • can be deleted if they are no longer needed.

26. VA chances charts Value added chances charts illustrates the likelihood that a learner has in obtaining each grade in a subject qualification, given his/her prior attainment. These chances are based on national data. A VA chances chart can be used: • as a predictive tool to indicate achievement based on prior attainment • as a motivational tool for target setting purposes • to assist in informing decisions in choosing appropriate learning programmes for learners and for supporting learners requiring additional support to meet their learning goals

27. The LAT Tutorial A tutorial on the LAT showing how to access VA and DT data relevant to your organisation is available at http://ffe.lsc.gov.uk/staff-development-resources/ If you are viewing this presentation while online, you can access the LAT tutorial directly by clicking on the button below Click here to go to the LAT tutorial

28. Section 5Recognising and recording progress and achievement

29. What is RARPA? RARPA aims to help you assure the quality of provision while focusing on learner achievement. Developed to help teachers, trainers and learners to record and recognise progress and achievement in provision which does not lead to nationally recognised qualifications or external accreditation.

30. The RARPA approach Based on five inter-related standards that help ensure your learners enjoy a high quality experience and maximise their success. Applies to learners on courses that do not lead to external qualification / certification The standards can be cross-referenced with one or more key questions in the Common Inspection Framework (CIF). The standards follow a logical sequence in line with recognised educational practice. Should not produce additional bureaucracy – effective quality assurance and quality management enables RARPA to be accommodated within current processes.

31. The 5 RARPA standards The RARPA standards refer to the five stages in a process: • Aims • Initial assessment • Identification of appropriately challenging learning objectives • Recognition and recording of progress and achievement during programme (formative assessment) • End-of-programme learner self-assessment, tutor summative assessment, review of overall progress and achievement

32. Who does RARPA benefit? Learners: It makes learning more effective by encouraging your learners to engage more actively by measuring their progress and achievements Teachers: By following the stages in the process, you and your colleagues can ensure that learning is taking place and that the learner is making progress Managers: By looking at evidence that the process is being followed, you can establish whether teaching, learning and assessment are effective Inspectors: The five stages are cross-referenced with the CIF, so evidence of implementing RARPA can also provide evidence of effective practice for Inspectors

33. Meeting RARPA standards Meeting RARPA standards means that you should: • apply RARPA processes in a way that benefits learners and makes sense to them • be open to change and to developing assessment practices as necessary • record evidence of learners’ progress in appropriate ways • help each individual learner realise her / his full potential

34. Section 6Further information

35. Further information on the Framework for Excellence • The Framework for Excellence website: www.lsc.gov.uk/ffe • The LSC website publications area: www.lsc.gov.uk/Publications/

36. Implementing measures of success: The Handbook • The Handbook contains an introduction to the Framework for Excellence and comprehensive details of: • what the measures of success are • explanations of the methodologies used • timetables for piloting and implementation • case studies.

37. Other sources of information • The Provider Gateway: LSC’s web-based internet portal which enables colleges and other providers to access information which supports self-assessment and quality improvement, available at: https://gateway.lsc.gov.uk/providergateway • The QIA’s Excellence Gateway: resources, materials, support and examples of effective practice, available at: http://excellence.qia.org.uk/vision

38. End of presentation © LSC November 2006 Published by the Learning and Skills Council. Extracts from this publication may be reproduced for non-commercial educational or training purposes on condition that the source is acknowledged and the findings are not misrepresented. This publication is available in an electronic form on the Learning Skills Council web site: www.lsc.gov.uk. Publication reference: LSC-P-NAT- xxxxxxx. If you require this publication in an alternative format or language please contact the LSC Help Desk: 0870 900 6800