1 / 34

Erwin Seyfried on behalf of CEDEFOP Version 12-01-2007

Cross Country Analysis on the Use of Indicators for Quality in VET Peer learning visit Helsinki, 13-15 December 2006. Erwin Seyfried on behalf of CEDEFOP Version 12-01-2007. Responses to Questionnaire. Participating countries host country + 16 peer countries = 17 Positive responses from

hollie
Download Presentation

Erwin Seyfried on behalf of CEDEFOP Version 12-01-2007

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Cross Country Analysis on theUse of Indicators for Quality in VETPeer learning visitHelsinki, 13-15 December 2006 Erwin Seyfried on behalf of CEDEFOP Version 12-01-2007

  2. Responses to Questionnaire • Participating countries • host country + 16 peer countries = 17 • Positive responses from • 13 countries: FIN; AT, CZ, DK, EE (only IVET), ES, IT, LT, NL, NO, PT, RO, SW (only CVT); • + 1 previously delivered paper from UK • Responses missing from • 2 countries: DE, SK

  3. Reminder: The coherent set of indicators • Indicator 1: VET-providers applying QA-systems • Indicator 2: investment in training of trainers • Indicator 3: unemployment acc. to groups • Indicator 4: prevalence of vulnerable groups • Indicator 5: participation in training • Indicator 6: completion of training • Indicator 7: destination after training • Indicator 8: utilisation of acquired skills • Indicator 9: responsiveness of VET-systems • Indicator 10: promotion of better access to VET

  4. Which indicators are used?

  5. …. indicators used?

  6. …. indicators used?

  7. Conclusion: Use of indicators • Usage of indicators 4, 5, 6 and 7 is most widespread • Indicators 8, 9 and 10 difficult to implement • Indicator 1 – big differences in implementation / in several countries required by law • Indicator 2 – different approaches; several sub-indicators • Linkage of indicators to QA crucial but not obvious • Whole set of EU indicators is used rarely (except FIN)

  8. Purposes / Aims Four main purposes which are linked to the relevant steps of CQAF: • Policy planning and target setting • Steering / monitoring and supervision • Measurement of effectiveness and efficiency • Decision making and performance-based funding

  9. Overall purposes / aims • Other, overall purposes/aims in using indicators: • Motivation of providers and staff • Continuous improvement of training delivery • Stimulation of excellence and innovation in VET

  10. Purposes in using indicators

  11. Purposes in using indicators

  12. Purposes in using indicators

  13. Conclusion: Purposes and aims • Usage of indicators for (budgetary) target setting and monitoring is most widespread • Indicators 3, 4, 5 most often used for planning and target setting • Most often mentioned as being important: indicators 6 and 7 • Measurement of effectiveness / efficiency and usage for performance-based funding is rare.

  14. (Problems with) Definitions • Indicator 1 • DK: descriptive use in IVET; obligatory use for evaluation of training courses in CVET • IT: Share of VET-providers having a quality certification (ISO class, EFQM, etc.) • RO: Capacity of a provider to offer VET programmes according to customers requirements and quality standards • Indicator 2 • a) financial definition: amount of funds invested in training of teachers and trainers (AT, ES, FIN, IT); financial target: EE • b) human resource definition: share of teachers and trainers completing further training (AT, EE, IT, NOR); amount of days (DK) or hours (IT) spent in training per capita and year • c) formal qualification: share of pedag. staff fulfilling formal qualification requirements (EE)

  15. ( Problems with) Definitions • Indicator 3 • Available in all countries through EUROSTAT; often even more detailed information on national level often available • Used mainly as background information only • Indicator 4 • Different vulnerable groups mentioned: gender, early school leavers, young unemployed people, long-term unemployed people, older people, handicapped persons, roma people (RO), level of education (IT), • Most often mentioned: migrants • Indicator 5 • Number of (individual) participants in IVT (AT, DK, ES, FIN, IT, LT, NL NO, RO) • Rates of population involved in CVT (AT, FIN, IT, LT, NL, RO) • Number of training programmes (CZ)

  16. ( Problems with) Definitions • Indicator 6 • Not standardised, not homogenous (AT) • Number of (annual) graduates / examinates (ES, IT, LT, NL, NO, SW), according to achieved marks (DK) • Completion within normative duration (FIN, RO) • Number of graduates according to (vulnerable) groups (CZ) • Three indicators used: share of fulfilled training places; share of graduates; drop-out rate (EE) • Indicator 7 • Employment after 2-4 months (EE), six months (EE, IT NO), 3/12/24 months (DK), 12 months (LT, NO, SW), 6/18 months (NL), 3,5 years (FIN), 6m/5 years (NO) • Number of entries in further /higher education (CZ, DK, EE)

  17. ( Problems with) Definitions • Indicator 8 • CZ: number of partners cooperating with VET-school • DK: opinion polls on usability from former students • DK, UK: opinion polls on usability from employers • SW: utilisation from perspective of former students • Indicator 9 - (descriptive information) • AT-proposal: minimal period to install new training programmes (till first graduation) • NL, NO: system for (tripartite) social dialogue with stake-holders • NL: satisfaction of stake-holders • UK: VET-providers demonstrate evidence of customised programmes meeting employers‘ needs • Indicator 10 - (descriptive information) • NL: efforts of VET-providers to increase accessability for vulnerable groups

  18. Conclusion: Definitions • Indicator 1: Big variations on provider level; information often descriptive • Common European core – but big national variations (for example for: Indicators 3, 5, 7, 8) • Limited possibilites for comparison of data on EU-level • Only soft definitions for indicators 8, 9, 10

  19. Collection of data

  20. Collection of data

  21. Collection of data

  22. Conclusion: Collection of data • Strong role of statistical records esp. for contextual information and output data (indicators 3, 4, 5, 6) – operational relevance for QA on provider level? • Provider reports often used as additional sources – reliability ? / comparability with other national (statistical) sources? • Difficulties with implementation of indicator 7 • Several research attempts to analyse data for indicator 8

  23. Availability of data by level (nat./reg./prov.) and time

  24. Availability of data by level (nat./reg./prov.) and time

  25. Availability of data by level (nat./reg./prov.) and time

  26. Conclusion: Availability of data • More information available on national / regional level then on provider level • Relationship between data on national, regional and provider level seems to be difficult - limited possibilities for comparison • Information for IVT much better than for CVT • Big variations in terms of timely availability

  27. Relevance of indicators • Indicator 1 • Mostly seen as relevant • Used for improvement of governance and accountability (NL) • Not primarely for assessment of quality • Many VET-providers use recognised QM-systems like EFQM, ISO etc • Indicator 2 • Often classified as relevant, but • Difficult to arrive at comparable data • Linkage to other indicators seems to be difficult • Restricted usability

  28. Relevance of indicators • Indicator 3 • Used mainly as background information • Relevant for better matching and shaping educational policies • Rarely applied on provider level although big (regional) differences • Used for social dialogue • Indicator 4 • Consideration of migrants classified as very important • Indicator mainly used as contextual information for policy development on national level • Partly the indicator is used to analyse participation and completion rates in relation to prevalence (FIN) • In some countries VET-schools receive addtional funding when serving vulnerable groups (FIN, NL)

  29. Relevance of indicators • Indicator 5 • Funding of VET-providers often depending on number of participants • Indicator 6 • Generally classified as very important • The only indicator which is (rarely enough) used for efficiency issues • Relevant also for financing (NL) • Completion rate needs to be desaggregated according to different (vulnerable) groups • Indicator 7 • Seen as very relevant for the matching issue • Used as basis for performance-based funding (FIN, SW) • Some countries rely on highly sophistaced national data collection systems • In other countries availability of data often depends on schools „handwork“ • Production of reliable data needs better coordination

  30. Relevance of indicators • Indicator 8 • Partly assessed within sectoral research (LT) • Very relevant for quality assurance and quality improvement • Indicator 9 • Partly assessed within sectoral research (LT) • Growing number of research on anticipation of skills needs • Many references to consultation with social partners and dialogue with other stake-holders • Indicator 10

  31. Conclusion: Relevance of indicators • All indicators of the European set of indicators are classified as being (very) important • Many of them are implemented in connection with educational reform or development strategies (CZ, EE, LT) • Clear indications to move sytems steering towards output and outcome indicators, giving high importance to indicators 6, 7 and 8) • In particular indicators 5, 6 and 7 are used for funding of VET-providers

  32. Additional remarks • In many countries much research, development and data analysis is undertaken to develop and implement the European and/or other indicators • Special research emphasis towards indicator 6 (completion), indicator 7 (destination), and indicator 8 (responsiveness) • Usage of indicators to be seen as part of an external (supervision) framework to improve quality of VET-providers

  33. Further national indicators • Number of trainees per trainer (EE) • Availability of ICT: number of PCs per trainee (EE) • Sustainable development (EE) • Student mobility • Drop-out rate (in the European set of indicators included implicitly as relation between participation and successful completion) • Participation of young people in skills competions (RO)

  34. Challenges for the European set of indicators • Promote usage of the whole EU set of indicators • Consider inclusion of additional indicators addressing • equity issues (esp. for migrants) • the efficieny of VET provision • Start discussions • on reliable methods of data-collection • how to make use of indicators in an external system for verification of quality? • how to make use of indicators on European level?

More Related