Dissemination of PSA Level 3 Evaluation December 2010

1. Dissemination of PSA Level 3 Evaluation December 2010

2. Agenda • Terms of Reference • Considerations • Evidence Base • Main Findings • Recommendations • Next Steps • Conclusion and Questions

3. Considerations …. • Outcomes- summary evaluation report plus four specialist reports • Reports issued to each college last term- follow- up inspection work in four colleges • Same team of specialist inspectors in each college • Evaluation Reports take due account of key policy drivers • Evaluation Reports well- populated with examples of good practice • Dissemination activities ….

4. Terms of Reference 1… • Scope of Evaluation- level 3 FE provision in- • ICT and computing • construction and built environment • electrical and electronic engineering • mechanical and manufacturing engineering

5. Terms of Reference 2… to inspect and report on • the quality of the curriculum for each priority skill area • the colleges’ strategic planning for the provision • the effectiveness of employer engagement and links with key stakeholders • the quality of the provision for learning • the standards of students’ work Also to • identify best practice • inform Department on impact of policies

6. 320 lessons Small number of lessons to address weaknesses in full-time students’ literacy, numeracy and ICT skills Sample of mandatory and optional units to improve technical literacy and numeracy skills College senior management teams curricular team leaders and course teams students Sample of employers Meetings with key stakeholders including DEL- Skills Strategy and Sectoral Bodies Evidence Base

7. Main Findings… overall effectiveness Overall effectiveness for each college- • it is very good one college • good in three • satisfactory in two Within each priority skill area: • Electrical and electronic engineering is good or better in 5 colleges- one is outstanding • ICT and computing it is good or better in 4 colleges-one is outstanding • Construction and mechanical and manufacturing engineering only good or better in half the colleges-the provision in mechanical and manufacturing engineering in one college is inadequate

8. Provision to meet needs of learners and local economy • Full-time provision fine in most colleges but….. • 2 year part-time craft and technician courses in nearly all of the priority skills areas is inadequate • 1 year or short part-time up-skilling level 3 courses including bespoke courses for industry varies across the priority skills areas and across the colleges • Significant gender gap with few female students recruited to full-time and part-time courses

9. STEM Expertise at a High Level…a critical success factor Fragmented curriculum planning across these areas results in gaps in provision and the underutilisation of resources Strategic Project Management Methodologies can/do make a difference Much more needs to be done to foster collaboration within and across colleges Middle managers have significant responsibilities Challenges of deploying best staff to meet different priorities Leadership and Management…1

10. Leadership and Management…2 • Quality Assurance arrangements are implemented systematically but are not sufficiently honed to identify and respond to weaknesses • The quality and management of specialist physical resources varies significantly across the sector • Management of specialist resources…a few examples of ineffective management of these resources which has an adverse impact on the students’ learning experiences particularly in construction and mechanical/manufacturing engineering

11. Leadership and Management..3 Economic Engagement • 2 colleges have well -developed strategies to support economic engagement across these areas- they are well-informed and linked effectively to the changes in the wider business environment including sustainable technologies • A common thread in both was the further development of extensive links with industry that had been gained over a lengthy period of time in one or more legacy colleges backed up with action to up-skill the expertise of the lecturing staff • In the other colleges there are examples of good practice in at least one priority skill area that are not shared effectively in the other areas

12. Leadership and Management..3 Economic Engagement • Nearly all of the colleges have well- developed links with local and regional employers and sectoral bodies- particularly at senior management level or through whole- college business support units • These links have a more positive impact when senior managers draw on the skills and expertise of those STEM middle managers and course teams who have well-developed links with industry • Some examples of good practice in those colleges who have been leading or have participated on projects funded under the Innovation Fund: Employer Support Programme • WDFs.. there were few examples of changes in their level 3 provision across these areas to meet identified skill needs. Examples of good practice ….

13. Quality of the Curriculum • In 4 colleges students have good opportunities to develop relevant technical and practical skills • In 2 colleges these opportunities are either limited or vary significantly across the priority skills areas-the curriculum is unduly theory-based and assessment led resulting in lower standards and high levels of student dissatisfaction • Most courses have a good blend of mandatory and optional units to develop the students’ technical literacy and numeracy skills • Additional qualifications can help but.. • College admissions criteria vary significantly and are not applied sufficiently rigorously

14. The quality of teaching and learning in the majority (69%) of the lessons is good or better and a significant minority (30%) are very good or outstanding A minority (27%) of the lessons are just satisfactory Most lecturers make use of ILT in their lessons but insufficient use is made of the more interactive features of educational technology to engage and motivate students One example of best practice in enhancing lecturers’ skills in making good use of ILT Quality of Teaching and Learning

15. Assessment • More work needed to enable course teams to work collaboratively across campuses in the design and moderation of coursework assignments • Some examples of good practice to reduce the assessment burden on students- integrated assignments • For most areas assessment planning is discrete and unit-based which results in an excessive assignment workload for students

16. CEIAG • Although students have good access to specialist careers information advice and guidance services in nearly all of the colleges more needs to be done to integrate this into the students’ main vocational units and through their timetabled tutorial programmes • The use of work-related learning to enhance the students’ knowledge and understanding of the world of work is inadequate in nearly all of the colleges

17. Student Support • The student tutorial programmes in nearly all of the colleges are not sufficiently well-organised to support students effectively in their learning particularly in reviewing their progress and setting targets for improvement • With the exception of 2 colleges electronic or paper-based student learner agreements provide little added value in helping students plan their work effectively • In some cases, course teams made more widespread use of other online reporting tools that they had developed to track the progress of their students • In the best practice(1 college) students have access to a well-designed tutorial system which gives good coverage to personal and social development-the college through its student support service has excellent arrangements in place to support students at risk of withdrawing from their courses

18. Standards and Outcomes..1 • The standards of the students’ work across the priority skills areas are mostly good in 4 colleges • In the other 2 colleges while there are pockets of good practice the standards of work in most of the priority skills areas are just satisfactory or inadequate • Most of the students achieve good standards of work in literacy and numeracy within their professional and technical units • Students can significantly enhance their literacy/ numeracy skills in their professional and technical units

19. Standards and Outcomes..2 • Well-planned project work relevant to the needs of industry can significantly enhance the standards of students’ work • Achievement rates vary significantly across the colleges and across the different priority skills area • Achievement rates for part-time courses are mostly higher than on full-time courses, where high levels of student drop out are a key weakness in a significant minority of courses

20. Ensure that the students enrolled on full-time courses have the relevant academic qualifications and interests to meet the challenges of the courses Promote these courses more effectively to female students, where enrolment levels are insignificant Links to other Evaluation Aims to improve the quality of learning to improve standards to improve the strategic planning to improve the quality of learning Actions required to bring about Improvement 1: Improve the quality of the curriculum across the Professional and Technical Areas

21. Foster more collaborative approaches to identify and exploit key connections across these PSAs to meet gaps in provision and make more effective use of specialist physical resources and to respond effectively to the needs of industry Links to other Evaluation Aims to improve the quality of the curriculum develop/ enhance employer engagement Actions required to bring about Improvement 2: Improve the strategic planning for the provision

22. Take action to address provision that is satisfactory or inadequate particularly in construction and mechanical/ manufacturing engineering where only half of the colleges have good or better quality provision Links to other Evaluation Aims to improve the quality of the curriculum to improve the quality of learning to improve standards Actions required to bring about Improvement 2: Improve the strategic planning for the provision

23. Ensure that human resource strategies are aligned more closely to curriculum planning particularly where course teams work in industry and in post-primary schools Develop more collaborative planning across the sector and within the colleges to widen student access to high quality specialist physical resources particularly expensive advanced manufacturing equipment Links to other Evaluation Aims to improve the quality of learning to improve standards to improve the quality of the curriculum develop/ enhance employer engagement to improve the quality of learning to improve standards Actions required to bring about Improvement 2: Improve the strategic planning for the provision

24. Make more effective use of the existing expertise of those lecturers and curriculum leaders with a strong background in STEM in curriculum planning, and in developing courses that meet the needs of industry and the NI workforce Develop more innovative approaches to promote and deliver part-time courses to meet the needs of the workforce to enhance their skill levels to meet the needs of industry Links to other Evaluation Aims to improve the quality of the curriculum to improve the strategic planning to improve the quality of learning to improve the quality of the curriculum to improve the strategic planning Actions required to bring about Improvement 3: Develop and enhance employer engagement and links with key stakeholders

25. Provide all full-time students with good opportunities to develop their technical and practical skills and to undertake work-related learning tasks to enhance their employability skills Links to other Evaluation Aims to improve the quality of the curriculum to improve the strategic planning develop/ enhance employer engagement to improve standards Actions required to bring about Improvement 4: Improve the quality of the provision for learning

26. Implement coherent strategies to improve student retention rates on those full-time courses with low levels of achievement Links to other Evaluation Aims to improve the strategic planning to improve the quality of learning Actions required to bring about Improvement 5: Improve the standards of students’ work

27. Next Steps … Publication of Reports Follow-up Inspections

28. Conclusion/ Questions….