SQA’s Approach to Quality Assurance of Assessment

2. SQA’s Approach to Quality Assurance of Assessment Matthew McCullagh Quality Manager Welcome 欢迎

3. Why are we here? • To introduce you to SQA’s New Approach to Quality Assurance • To help you understand the key changes to Quality Assurance process • To help you become familiar with: • The new Quality Assurance Criteria • The new Confidence Statements • How visits are planned, conducted and reported

4. Thursday 6th September 2012 9.00 Coffee and Registration 9.30 Introduction, New Approach to QA Overview, Aims and Objectives of Training Days 10.10 Preparation for Qualification Verification Activity, Attitudes and Behaviours, KPMs 11.00 Break 11.15 Preparation for Qualification Verification Activity, Attitudes and Behaviours 12.00 Planning for Qualification Verification Activity – the theory 12.30 Lunch 1.30 Conducting the Qualification Verification Visit – the theory 2.45 Break 3.00 Introducing the Quality Assurance Criteria 3.20 Quality Assurance Criteria Workshop 4.30 Close

5. Friday 7th September 2012 9.00 Coffee and Registration 9.30 Feedback: The Quality Assurance Criteria 11.15 Break 11.30 Decision Making 12.30 Lunch 1.30 Communicating Decisions 2.00 Theory of Completing Excel Reports/Action Plans 2.30 Break 2.45 Writing effective Verifier Reports 3.45 Case Study Assignment 4.00 Summary and Q & A 4.30 Close

6. SQA’s Approach to QA • SQA is moving to introducing a new approach to Quality Assurance of qualifications delivered in China • It is based on SQA’s Principles of Quality Assurance: • Open, fair and transparent • Aimed at risk reduction • Proportionate • Intelligence led • A shared responsibility • Constantly reviewed and improved

7. What is the new approach to Quality Assurance SQA now operates a transparent, intelligence-led, risk based model for quality assurance of approved centres and potential centres delivering our HND qualifications in China.

8. Timeline of Implementation of New Approach in China May 2012 September 2012 September 2011 March 2012 September 2012-2013

9. New Quality Criteria • Five Categories: • Management of a centre • Resources • Candidate Support • Assessment & Verification • Records/Data Management • All signposted to four QA processes • All pre-rated as High/Medium/Low impact • All supported by possible sources of evidence

10. The New Approach to Quality Assurance: The Four Processes • Systems Approval • Qualification Approval • Systems Verification • Qualification Verification

11. High/Medium/Low Impact Criteria • All criteria are important • However, some are more important than others • Those that are most important are rated as High Impact criteria • Those that are less important are rated as Low Impact criteria • In between these two are medium impact criteria. • Failure to meet a High Impact criteria will have a greater impact on the outcome of the visit

12. Sufficiency of evidence • No Systems Verifier or Qualification Verifier can change the impact rating of a criteria. • For each criterion you verify, you must decide whether the centre has presented you with: • Sufficient evidence to meet the criterion • Some, but insufficient evidence to meet the criterion • No evidence to meet the criterion

13. Good Practice and Guidance • As part of the verification visit, you should identify good practice that is evident in the centre. • You should also give advice and guidance to help centre staff improve the delivery of our qualifications • This is a mandatory part of your role as a Qualification Verifier

14. Sufficiency of evidence • We will support you and help you to understand what sort of evidence centres can provide to meet each criterion • Once you have made your decision regarding the sufficiency of evidence, SQA staff will then calculate our level of confidence • This will depend on the impact level of the criteria and the sufficiency of the evidence presented

15. Introduction of Confidence Statements

16. How Confidence Statements are calculated • Confidence statements are given for the outcome of each Systems Verification and Qualification Verification visit for each of the categories of criteria verified • These are based on the sufficiency of the evidence provided to the you • A summary statement of confidence is also given, based on the confidence statements for each category

17. What if non-compliances are identified? • Where there is No evidence or insufficient evidence to meet a criterion, you must agree with centre staff, during the visit: • the action the centre needs to take in order to become compliant. • The evidence they need to produce • Where to send this evidence • The date by which the action must be taken

18. What happens then? • SQA will ask you to consider the evidence the centre sends us and ask you to consider whether it is Sufficient or insufficient. • Depending on the sufficiency of the evidence, the Confidence Statement will be re-calculated.

19. Sanctions • If the outcome of a verification visit results in a Confidence Statement of Reasonable, Minimal or No Confidence, SQA staff may decide to place a sanction on a centre until our confidence increases to Broad or High. • Our confidence may decrease if a centre fails to meet action points agreed during a verification visit. • Sanctions will only be used where necessary.

20. Sanctions

21. Benefits of new approach for centres • Open and transparent to all • Clear guidance available to verifiers • Proportionate response where issues are identified • Focused on Good Practice and improvement as well as compliance

22. Questions

23. SQA’s Approach to Quality Assurance of Assessment David Pirnie Lead Verifier Welcome 欢迎

24. Qualification Verification The Qualification Verifier role activities Prepare for verification Plan verification Carry out verification Communicate verification decision Complete verification report

25. Preparing for Qualification Verification The Qualification Verifier role activities Prepare for verification Plan verification Carry out verification Communicate verification decision Complete verification report

26. Being prepared “The best preparation for good work tomorrow is to do good work today”. Elbert Hubbard

27. Preparing for Qualification Verification Before carrying out any activities, verifiers are required to have a full understanding of their role and responsibilities. This involves acquiring knowledge and keeping it up to date.

28. Preparing for Qualification Verification Required knowledge – three key areas: Knowledge of your subject area Centre roles and centre responsibilities Qualification Verification

29. Preparing for Qualification Verification Knowledge of your subject area: Have a common interpretation of the Unit Standards including the Evidence Requirements for each unit you have been assigned to verify Know how to access Arrangements Documents, Instruments of Assessment and Marking Schemes to gain a full understanding of content

30. Preparing for Qualification Verification Knowledge of centres roles and responsibilities: Centre Co-ordinators Assessors Internal verifiers

31. Preparing for Qualification Verification Knowledge of the Qualification Verifier role: Role Profile Sources of support Verification policies and procedures Values

32. QIIPS:Quality, Integrity, Innovation, Partnership & Service Sounds like: “QUIPS” – a dictionary definition: “A clever remark often prompted by the occasion”.

33. SQA Values Quality Integrity Innovation Partnership Service

34. SQA Values Quality As custodians of the standards we verify, we continually strive to maintain consistent quality in their assessment.

35. SQA Values Integrity We use open and honest communication with internal and external customers (centres), promoting transparency, and building trust with others. We take a standardised approach to verification as laid down by SQA and agreed by consensus within our Qualification Verification team, even if our personal opinions may at times conflict.

36. SQA Values Innovation We view unfamiliar approaches to assessment and verification with an open mind, whilst ensuring quality is maintained. We believe that creativity should be seen not as a threat to quality assurance but as an opportunity for improving upon existing practice.

37. SQA Values Partnership We understand that by working in partnership with SQA centres, we will achieve common goals of excellence and consistency in assessment.

38. SQA Values Service We maintain a professional approach with SQA centres at all times, regardless of the challenges we might face in carrying out our work.

39. Preparing for Qualification Verification More information on the preparation phase of Qualification Verification can be found in the first section of SQA’s New Approach to Quality Assurance: Guidance on Visiting Verification For HN Qualifications in China

40. Workshop Values Exercise

41. Planning for Qualification Verification The Qualification Verifier role activities Prepare for verification Plan verification Carry out verification Communicate verification decision Complete verification report

42. Planning “Failing to plan is planning to fail”. Alan Lakein

43. Visit Planning: Sampling SQA cannot always verify all HN Units, for all candidates, on every visit to a centre, therefore a sampling strategy has to be adopted. Purpose of sampling: To establish that there is a common interpretation of the standards within each centre and that the assessment and verification systems allow valid, reliable and fair assessment decisions to be made.

44. Visit Planning: Applying a sampling frame SQA is responsible for systematically sampling on a percentre basis: Selected Units Assessment decisions and practice Methods/Instruments of assessment Problematic/revised units Locations where assessment takes place Evidence of candidates across Units Documented evidence of policies/procedures (implementation)

45. Visit Planning SQA (China Office) agrees optimum date (s) for visiting verification. This is an agreement between SQA, each centre and each QV SQA selects Units to be verified and communicates this to each centre (copying to SQA China Office) SQA (China Office) confirms to each QV, the centre(s) they should visit, the date of each visit, and the Units they should verify Qualification Verifier contacts centre to confirm visit date(s) and travel arrangements

46. New Approach to Quality Assurance Welcome Back 欢迎回来

47. Carry out Qualification Verification The Qualification Verifier role activities Prepare for verification Plan verification Carry out verification Communicate verification decision Complete verification report

48. Carry out Qualification Verification Discussion point Consider the verification activities you perform from the time of arrival until the end of the visit. Contribute to the discussion by letting us know what kind of techniques/approaches you have used that you have found particularly beneficial.

49. Carry out the visit Prepare yourself for the visit Agree agenda and running order of visit Conduct sampling activities Judge the sufficiency of evidence

50. Conduct sampling activities Complete Units (all Outcomes complete) Incomplete Units (some Outcomes complete) Evidence not yet internally verified Assessment/verification decisions Assessment and verification practice Interviews with assessors/verifiers/candidates Referencing of evidence to standards