Indicators&Criteria in External Quality Assessment Lithuanian experience

www.skvc.lt Indicators&Criteria in External QualityAssessment Lithuanian experience Almantas Šerpatauskas, Acting Director Centre for QualityAssessmentinHigherEducation (SKVC)

Objectives • to contribute to creation of favorable conditions for free movement of persons • assessment of foreign qualifications • provision of info and consultations • oversight of HEI recognition practices • to encourage quality of activities of HEI • review and accreditation of HEI (institutional review) • evaluation and accreditation of HE study programs (ex-ante and ex-post) • assessment of (new) establishing HEI

SKVC asQA agencyhasknowledge and experienceinapplication and interpretationoftheESG, and canassistincapacitybuildingofrelevantauthoritiesindevelopingcountriessincewe: operateasexternal QA agencyin Lithuania for already 23 years; successfullyunderwenttwoexternal international reviews, organisedby ENQA (in 2012 and 2016); are listedastrustworthy QA agencyon EQAR from 2012, didreviewsabroad; workasexpertsof QA invariouscapacities (servingonBoardsof ENQA and CEENQA; asexpertsinENQA reviewsofagencies to determinetheircompliance to ESG; as Bologna experts; asexpertsinTwinningprojects, asmembersof international and localworkinggroups; asinvitedspeakers to variousinternationalevents; asconsultants to various QA agencieswhich are intheprocessofdevelopment). Expertiseof SKVC in QA of HE 9

Ourownquality SKVC asacademicinformationandrecognition centre: voluntaryself-evaluation and externalreviewbythe international panelfollowing SQUARE methodology (endof 2015 – beginning 2016). SKVC asexternalqualityassuranceagency: Mandatoryself-evaluationandexternalreviewagainstStandardsandGuidelines for QualityAssuranceintheEuropeanHigherEducationArea (ESG) by ENQA: 1stin 2012, 2nd – in 2016.

Internationalrelations • INQAAHE (International Network for Quality Assurance Agencies in Higher Education) – full member from 1997 • ENIC/NARIC (European Network of National Information Centres / Network of National Academic Recognition Information Centres) – full member from 1999 • CEENQA (The Network of Central and Eastern European Quality Assurance Agencies in Higher Education) – co-founder and full member from 2000 • ENQA (associate member from 2000, full member since 14 Sept 2012) • Listed on EQARastrustworthy QA agencyfrom2012

SKVC’s commitment to ENQA • ENQA GA in Vilnius (October 2013) • Former SKVC Director – ENQA Vicepresident • Two members of staff – on ENQA Board (in the past) • Two members of staff – among ENQA reviewers for QA agency reviews • Participation in working groups: • onimpactofqualityassurance • staffdevelopment • stakeholderinvolvementinqualityassurance • excellence • reviewoftheexternalreviewmodel for qualityagencies • internal quality assurance and complianceofqualityagencieswith ESG requirements • links betweenqualityassurance and qualifications‘ recognition

External Evaluation of study prorammes 2011-2018 • All study programmes must be evaluated at least once every 6 years; • Evaluation and accreditation of new study programmes and study programmes in operation; • I & II cycle study programmes, non-degree programmes – evaluation by SKVC; • III cycle studies – evaluation by Research Council of Lithuania;

Study programmes in operation • All study programmes are subject to evaluation and accreditation • Accreditation is based on study programme evaluation report.

Evaluation of study programmes in operation • Study programmes of the same (or similar) study field are evaluated at once; • SKVC informs HEIs when their programmes going to be evaluated at least a year before submittion of self-evaluation report;

Evaluation process Follow-up Appeal Consideration of the final report and publication Writing an evaluation report Site visit Preparation for external evaluation Self-evaluation Planning of evaluation

Expert panels • International or local • Composition of panels: academics, employers and students

Expert panels Number of study programmes evaluated by …

Study programme evaluation areas

Evaluation criteria Each evaluation area must be evaluated according to set of criteria. Eg.Programme aims and learning outcomes: • the programme aims and learning outcomes are well defined, clear and publicly accessible; • the programme aims and learning outcomes are based on the academic and/or professional requirements, public needs and the needs of the labor market; • the programme aims and learning outcomes are consistent with the type and level of studies and the level of qualifications offered; • the name of the programme, its learning outcomes, content and the qualifications offered are compatible with each other.

Evaluation area, criteria & indicators

Study programme accreditation The study programme evaluation is positive or negative. The decision on study programme accreditation is based on external evaluation report. • Study programme can be accredited for: • 6 years • 3 years • The programme can be given non-accreditation!

Evaluation framework to be introduced since 2019 Evaluation and accreditation of study fields instead of individual study programmes. Use of data gathered from national registers, Monitoring agency, Research Council. Importance of scientific level in the field. Simplified vs full procedure

Indicators • Threshold and monitoring indicators: • Scientific level in the field; • Student competitive score in admission; • Student drop-out in 3 years period; • Percentage of teaching staff working in HEI on full time basis; • Percentage of graduates employed in relevant positions 12 months after graduation; • Percentage of unemployed graduates 12 months after graduation; • Ratio of incoming and outgoing exchange students in 3 years period; • Percentage of international students in the study field in 3 years period; • Ratio of incoming and outgoing exchange teaching staff in 3 years period; • Percentage of study programmes accredited for 6 years; • Percentage of study programmes accredited for 3 years twice in a row; • Student satisfaction about quality of studies; • Number of students in the study field in 3 year period; • Percentage of students finishing their studies on time.

Agency’s point of view • Quantitative and qualitative indicators. • Data must be interpreted in wider context!

Thank you! almantas@skvc.lt www.skvc.lt

Indicators&Criteria in External Quality Assessment Lithuanian experience