Maximizing Support for More Able and Talented Learners: Tracking, Monitoring, and Evaluation Workshop

1. MORE ABLE AND TALENTED SUPPORT FOR FE CPD PROGRAMME  WORKSHOP 3 TRACKING, MONITORING AND EVALUATION

2. 2 WELCOME, INTRODUCTIONS AND PURPOSE OF THE WORKSHOP

3. 3 ACTIVITY 1 WHOLE GROUP DISCUSSION • What are your expectations of the workshop? • What do you need and want to know about tracking, monitoring and evaluation of the support provided to MAT learners? • What do you want to achieve today?

4. 4 SCENE SETTING WORKSHOP OVERVIEW This workshop will explore: • Setting appropriate key performance indicators (KPIs) and targets for MAT delivery; • Tracking and monitoring learner progress across the institution that ensure that MAT learners are accommodated; • Ensuring that evaluation is used to plan further MAT provision and support.

5. 5 THE OBJECTIVES OF THIS WORKSHOP ARE: • To secure alignment between the institution’s overall approach to supporting MAT learners, and its systems for target-setting, tracking, monitoring and evaluation. • To review current institutional KPIs and targets, and identify opportunities to refine and adapt them to report specifically on MAT learner outcomes. • To consider how learner tracking and monitoring systems can be used to track the progress of MAT learners.

6. 6 THE OBJECTIVES OF THIS WORKSHOP ARE: • To develop systematic approaches for evaluating the support given to MAT learners, including effective involvement of learners, parents/carers and stakeholders. • To agree team and personal action plans to secure consistent, effective tracking, monitoring and evaluation of the support given to MAT learners.

7. 7 WORKSHOP OBJECTIVE 1 To secure alignment between the institution’s overall approach to supporting MAT learners, and its systems for target-setting, tracking, monitoring and evaluation.

8. 8 ACTIVITY 2 PAIRED DISCUSSION What does the term More Able and Talented (MAT) mean to you? Do you know who your institution’s MAT learners are? How? What are your institution’s main initiatives and activities to support MAT learners? Plenary

9. 9 SETTING EFFECTIVE KPIS: CONSIDERATIONS • Ensuring that indicators are inclusive and reflect the whole learner population • Reflecting academic, vocational and other types of ability • Availability, timeliness and accuracy of data on MAT learners and their achievements • Measuring impact as well as inputs or processes.

10. 10 EXAMPLES OF MAT PERFORMANCE INDICATORS Process indicators • % of staff trained in identifying and supporting MAT, of which x are fully bilingual • Staff levels of confidence in identifying and supporting MAT • % of staff confirming awareness of the MAT strategy • % of learners engaged in mentoring relationships of which x are fully bilingual • % of vocational learners identified and recorded as MAT • % of academic learners identified and recorded as MAT.

11. 11 EXAMPLES OF MAT PERFORMANCE INDICATORS Outcome indicators • Attainment of outstanding grades (A*, A, Distinction* and equivalent) • Value added scores • Entrants/achievements in skills competitions • Entrants/achievements in other competitions (sports, arts, debating, enterprise) • % of learners progressing to ‘Sutton Trust 30’ universities, or to appropriate alternative routes for learners who wish to progress to Welsh-medium HE • % of learners progressing to Higher Apprenticeships or to selective industry recruitment schemes • Learner feedback on enrichment activities • MAT learner and parent satisfaction levels.

12. 12 ACTIVITY 3 SMALL GROUP DISCUSSION What data does your institution currently collect about MAT learners? When and how is this collected? Where is it recorded? Are there any gaps? Plenary

13. 13 WORKSHOP OBJECTIVE 2 To review current institutional KPIs and targets, and identify opportunities to refine and adapt them to report specifically on MAT learner outcomes.

14. 14 • Institutional MAT strategy and initiatives • Overall institutional KPIs and targets MAT KPIs and targets MAT KPIs and targets need to…

15. 15 …SUPPORT THE DELIVERY AND EVALUATION OF YOUR INSTITUTION’S MAT STRATEGY AND INITIATIVES • Does your institution have an overall strategy, plan or set of aims for supporting MAT learners? • What KPIs could help you to know whether it’s working? • What MAT initiatives does your institution deliver e.g. Seren, skills competitions, sporting excellence, entrepreneurship? • What measures or targets are associated with these?

16. 16 …FIT WITH YOUR INSTITUTION’S OVERALL SET OF KPIS AND TARGETS, AND ARRANGEMENTS FOR REPORTING ON THESE • How do you measure learner participation, the quality of teaching and learning, successful outcomes? • Do these measures lend themselves to adaptation to include MAT outcomes? • What KPIs or targets do you already have for MAT learners? Are these comprehensive or is anything missing?

17. 17 USING THE POST-16 CONSISTENT PERFORMANCE MEASURES ACHIEVEMENT The completion and attainment of learners undertaking A level, vocational and Welsh Baccalaureate programmes. • % of learners achieving three A*-A grades • % of learners achieving Distinction grades for vocational qualifications MAKE IT WORK FOR MAT VALUE ADDED The progress made by learners, over and above what would normally be expected based on their prior attainment and gender. • % of learners achieving a positive Average Grade Per Entry score DESTINATIONS The proportion of learners who progress into further learning (including higher education) and/or employment. • % of learners progressing to Russell Group/Sutton Trust 30 universities • % of learners progressing to Higher/ Degree Apprenticeships

18. 18 ACTIVITY 4 SMALL GROUP DISCUSSION Group 1: What is the institution’s MAT strategy (and/or individual MAT initiatives and programmes) trying to achieve? Group 2: What are the institution’s main KPIs and targets for learners and learning outcomes? (Resource 2) Feedback and plenary: how well do the two align?

19. 19 WORKSHOP OBJECTIVE 3 To consider how learner tracking and monitoring systems can be used to track the progress of MAT learners.

20. 20 THE BEST APPROACHES TO TRACKING (1) • Include real-time information, or as close to it as possible • Are accessible and easy for teachers, tutors and other staff to use • Involve learners themselves in monitoring their progress and setting targets • Allow early identification of concerns and problems.

21. 21 THE BEST APPROACHES TO TRACKING (2) • Support tracking and measurement of progress towards institutional targets • Enable timely reporting to senior leaders and others • Are secure and limit access to personal and sensitive information to those who really need to know • Should allow you to analyse the progress of different groups of learners, including MAT learners.

22. 22 MAKING IT WORK FOR MAT LEARNERS - CONSIDERATIONS • At what stages in the learner journey are MAT learners identified? • How and when is learners’ MAT status recorded in your tracking systems? • What key milestones or targets are, or could be, recorded for MAT learners? e.g.: Seren participation, university applications, skills competition entry • How does your tracking system enable you to measure progress towards MAT targets (such as achievement of higher grades)?

23. 23 ACTIVITY 5 SMALL GROUP DISCUSSION Map out the main elements of your learner tracking and monitoring system, and identify how MAT considerations could be integrated into it. • How and when are MAT learners identified? • How and when are staff able to record information about learners’ progress, goals, and any barriers to achieving those goals? • Does the system need to change to enable you to do this more effectively? • What information about MAT learners needs to be available to tutors, course teams, senior leaders, governors, others? Plenary

24. 24 MONITORING MAT LEARNERS • Monitoring should not rely solely on numerical data but be informed by professional input from practitioners • Learners’ (and parents’/carers’) own perspective on their progress should be a key consideration • There should be a ‘feedback loop’ so that learners and practitioners can evaluate whether MAT-related activities and support are working • There should be flexibility to adjust approaches and targets as the learner progresses.

25. 25 ACTIVITY 6 WHOLE GROUP DISCUSSION What are the barriers to effective tracking and monitoring of MAT learners, and what could be done to address these? Consider: • The availability and flow of information on MAT learners • Whether infrastructure/software needs to change • Staff skills, confidence and engagement • How to ensure that users (staff and learners) see the system as something that adds value for them, rather than creating bureaucracy Plenary

26. 26 WORKSHOP OBJECTIVE 4 To develop systematic approaches for evaluating the support given to MAT learners, including effective involvement of learners, parents/carers and stakeholders.

27. 27 EVALUATING THE IMPACT OF MAT SUPPORT • Evaluate the strategy as a whole, not just individual MAT initiativesor projects • Build in your evaluation criteria and processes from the outset • Evaluate the strategy as a whole, not just individual MAT initiativesor projects

28. 28 ‘DESIGNING IN’ EVALUATION • What baseline information do you have to compare the ‘before’ and ‘after’ positions? • When and how will you evaluate impact? • What evidence will you need? • How will you know if the strategy has been successful? • How will you share and build on successes? • How will you learn from mistakes or failures? • What is the process for making changes to your approach if you need to, and who will make decisions on this?

29. 29 ACTIVITY 7 SMALL GROUP DISCUSSION How do we currently evaluate impact – use Resource 3 to record your answers to these diagnostic questions EVALUATINGIMPACT How do we collect and use baseline information for comparative purposes? What evidence do we need to collect? How are parents/ carers, learners and stakeholders Involved? When do we evaluate impact? How do we know if the MAT strategy has been successful? What is the process for making changes if needed, and how long will this take? How is feedback incorporated and disseminated?

30. 30 INVOLVING LEARNERS, PARENTS/CARERS AND STAKEHOLDERS IN THE EVALUATION PROCESS • Consider how your existing approaches, such as learner voice, can be tailored to gather specific information on MAT experiences • KPIs should include indicators relating to feedback from stakeholders, such as levels of satisfaction with MAT activities • Ensure that stakeholders and partners are clear on what you are trying to achieve together: does everyone have the same objectives? • Build in feedback on an ongoing basis, so that you can make changes and improvements where necessary.

31. 31 ACTIVITY 8 SMALL GROUP DISCUSSION What would effective MAT support look like from the point of view of: • Learners? • their parents/carers? • Universities? • Employers? What evidence would you need to evaluate this? Plenary

32. 32 WORKSHOP OBJECTIVE 5 To agree team and personal action plans to secure consistent, effective tracking, monitoring and evaluation of the support given to MAT learners

33. 33 ACTIVITY 9 WHOLE GROUP DISCUSSION Using Resource 4 (sections 1 and 2) consider: The wider institutional/departmental actions and CPD needed to help support the tracking, monitoring and evaluation of the support given to MAT learners.

34. 34 ACTIVITY 10 INDIVIDUAL REFLECTION Recording your CPD needs – personal ‘to do’ list Using Resource 4 (section 3) record your individual actions and CPD needs which have emerged through this workshop. Reflect on and decide: • one person you need to speak to or liaise with • one thing you need to find out • one thing you need to do.

35. 35 WORKSHOP EVALUATION • What worked well? • Even better if?

36. 36 Facilitator next steps THANK YOU