Oman Academic Accreditation Authority (OAAA)

1. Oman Academic Accreditation Authority (OAAA) Briefing onInstitutional and Program Standards Assessment: Approach to decision-makingDr Salim Radhawi11 March 2014

2. Briefing Outline OAAA • Evolution of Institutional and Program Accreditation • HEI and Program QA processes • Project aims and outcomes • Features of standards and assessment processes • Consultation process and feedback • Organisation of standards, ratings and outcomes • Public Reporting and HEI comparability • Fees for Standards Assessments

3. Evolution of Institutional and Program Accreditation (1) OAAA • HEI and program accreditation are key parts of OAAA’s mandate (Royal Decree 54/2010) • First set of institutional and program standards were published in ROSQA in 2004 • ROSQA standards used in 2004/2006 (two HEIs went through institutional and program accreditation). • A review of ROSQA was carried out in 2006; this was followed by the development of the Quality Plan

4. Evolution of Institutional and Program Accreditation (2) OAAA • A new two-stage approach was proposed for HEI accreditation (Quality Audit and Standards Assessment) • The first Quality Audits were carried out in 2008 • A separate process for program accreditation was proposed • The Institutional Standards Assessment project was originally launched in 2011 • The current stage of the project, which includes both Institutional and Program Standards Assessments, commenced in 2013

5. HEI QA Processes First cycle commenced HEI Accreditation 2008 HEI Licensure Stage 1 : Quality Audit 4 years ≤4 years Appeal HEI Accreditation HEI Accreditation Met Certificate Stage 2 : Standards Assessment Not met KEY 1-2 years on Probation Start/End Met Process HEI Accreditation Terminated HEI Standards Reassessment Document OAAA

6. Program QA Processes OAAA

7. Projects aims and outcomes OAAA • To review and revise • HEI standards; program standards; and accreditation processes as set out in ROSQA to improve their relevance to the Omani context and to reflect current regional and international best practice (through benchmarking) • To produce • Institutional and program Conceptual Design Frameworks (CDFs) with clear approach to decision-making • a revised set of HEI standards and a revised set of generic standards for programs • Institutional and Program Standards Assessment Manuals which include guidelines for HEIs and external reviewers • Training workshops for the sector

8. Features of HEI and Program Accreditation OAAA • Recognition that responsibility for quality lies with HEIs • Based on ROSQA but incorporates criteria not included in ROSQA, e.g. academic integrity • Align with the nine areas of the scope of the QAM and reflect quality audit findings • Internationally benchmarked and reviewed • Acknowledge the diversity of HE provision in Oman • Encourage excellence – HEIs can receive Accredited with Merit overall or at standard(s) level • Encourages the use of ADRI • Developed through a consultative process

9. Features of the Institutional Standards Assessment process The process looks at all areas of activity within an HEI and considers the quality systems applied to programs National institutional accreditation is compulsory for all HEIs Takes into consideration the HEI’s response to formal conclusions in Quality Audit Report National schedule based on audit schedule with some flexibility OAAA

10. Features of the Program Standards Assessment process The program is the unit of analysis but considers the impact of institutional-level activities National program accreditation is compulsory for all HEI programs Accreditation is for Omani programs as well as programs developed by overseas HEIs and/or with external accreditation Program accreditation can be applied for once a cohort of students has graduated OAAA

11. Consultation Process Formation of a national Consultative Committee to facilitate comprehensive consultation with the HE sector Formation of an external panel of experts to provide an international perspective Working in partnership with MoHE Dialogue with external stakeholders (including Education Council, MoM, MoH, MoD, and MoE) Publication of drafts on OAAA website with discussion board National Symposium October 2013 (to which CC, stakeholders, professional body, employer and student representatives were invited) OAAA

13. Response to Feedback:Organisation of the Standards OAAA • A considerable amount of feedback highlighted the large number of elements to be assessed • There are nine standards • There are 75 criteria • Each criterion has a number of indicators • Indicators no longer need to be met for a criterion to be met – indicators are guidelines • Rating performance and commentary against indicators is no longer required • HEIs may provide other evidence to show that the requirements of a criterion has been met

14. Response to Feedback:Approach to Decision-making OAAA • Significant feedback was provided on the rating of ‘not applicable’ (NA) with a numerical value (‘0’) • No value is now given to NA • Feedback suggested that the rating scheme was too subjective and difficult to apply • Clearer definitions have been provided for each criterion rating • Descriptions/characteristics will assist HEIs/reviewers in determining the most appropriate rating for each applicable criterion and standard

15. Positive Feedback OAAA [The CDFs] are in many respects excellent, so congratulations to you all. Prof Malcolm Cook, Former Pro-VC, University of Exeter, UK In my view the Conceptual Design Framework is in very good shape. Prof Ulrich Hommel, Director, EBS Business School, Germany It was real It was a pleasure for me to join your national symposium.  I was really impressed with the approach OAAA has taken and the way participants got engaged in it. Prof Badr Abu Ela, Executive Director, CAA, UAE I think you should submit both the policies/standards and processes for international recognition. Dr Mike Hillyard, Former President of the University of St Augustine, USA

16. Previous and Current Approach to Organisation of the Standards OAAA

17. Comparison of Previous and Current Criterion Rating Scheme OAAA * These terms were revised following further benchmarking and in order to improve clarity of understanding

18. ADRI as a Review Tool IMPROVEMENT APPROACH RESULTS DEPLOYMENT OAAA Internal ADRI Review Followed by External Review

19. Criterion Rating Definitions OAAA

20. Example of ‘Excellent’ OAAA • ‘Excellent’ rating can be compared to ‘Commendation’ in Quality Audit Reports A revised innovative academic advising system was successfully implemented and has been shown to have had a significant impact on retention and identifying ‘at risk’ students; the system has received significant positive feedback, been periodically reviewed for effectiveness, indicating continuous improvement Rating against criterion 6.5

21. Criterion Rating Definitions OAAA

22. Example of ‘Good’ OAAA • ‘Good’ rating can be compared to positive text in Quality Audit Reports As part of its continuous improvement system, the HEI has introduced a revised academic advising system in response to feedback; there is a comprehensive handbook and training for staff and students which has been consistently implemented; the system has had a positive impact; and the HEI has clear plans for how the system will be evaluated Rating against criterion 6.5

23. Criterion Rating Definitions OAAA

24. Example of ‘Satisfactory’ OAAA • ‘Satisfactory’ rating means that the HEI/program has shown that it meets the requirements of the criterion The HEI has implemented an effective formal academic advisory system which, overall, supports students in meeting their educational goals; the system has been evaluated and improvement plans have been implemented in most departments Rating against criterion 6.5

25. Criterion Rating Definitions OAAA

26. Example of ‘Not met’ OAAA • ‘Not Met’ rating could be compared to an Affirmation or Recommendation in Quality Audit Reports. While the HEI has a policy for academic advising, the system has not been implemented effectively throughout the HEI and has not been monitored or evaluated for its effectiveness Rating against criterion 6.5

27. Examples of ‘Not applicable’ OAAA • ‘Not Applicable’ rating needs to be justified by the HEI/program From XX HEI: Criteria 3.1 – 3.6 do not apply to XX HEI as it is classified as a College and does not run Student Learning by Research Programs (see SM001 HEI license approval)

28. Criteria ratings towards meeting the standard OAAA * ‘Most’ in this context means more than 50%

29. How Standards Ratings inform the Assessment Outcome OAAA

30. Standards Assessment Outcome: Outcome Deferred OAAA

31. Status of HEIs and Programs on OAAA website Licensed Audited (for HEIs only) On probation Accredited Accredited with Merit Accredited with Merit in one or more standards Not accredited (after Standards Reassessment) OAAA

32. Public Reporting and Comparability Accreditation outcomes will be published on OAAA website, with ratings against each standard and criterion A 4-point rating scale provides a transparent means for stakeholders to identify/compare how an HEI/program has performed Stakeholders will be able to apply their own priorities in order to identify the HEI/program which meets their needs This approach to public reporting avoids institutional/program ‘league tables’ OAAA

33. Stakeholder Comparison of HEIs OAAA Overall score HEIs

34. Stakeholder Comparison of Programs OAAA Stakeholders input their own weightage for every criterion Overall score Programs

35. Fees for Standards Assessments OAAA • A proposal for fees for Institutional Standards Assessment has been submitted to the Ministry of Finance • The fee will be calculated on the HEI’s Classification and number of FTE students • The fee structure is based on international benchmarking • Fees for Program Standards Assessment are yet to be determined

36. www.oaaa.gov.om