The Importance of Self-Study Reports

1. The Importance of Self-Study Reports Richard Lewis Former Pro-vice-chancellor of the Open University (UK) Immediate Past President of the International Network of Quality Assurance Agencies in Higher Education (INQAAHE) Imp of SS Reports KSU Feb 11

2. Outline of Presentation- 1 The Importance of Self Study Reports cannot be over emphasised. Relevant to both Institutional and Programme Accreditations and Reviews. Why they are so valuable. Similarities and differences in the approaches taken by Quality Assurance Agencies What they should cover. Imp of SS Reports KSU Feb 11

3. Outline of Presentation- 2 Some technical issues about their presentation. Some theoretical issues about why self study reports should not be seen as stand alone documents but as by-products of the normal operating systems – or in other words part of the quality culture. Imp of SS Reports KSU Feb 11

4. The Vital Importance of (Institutional) self Studies They must inform the team of the nature of the institution, its distinguishing features, key institutional data, the results of its self-critical review of its activities and the steps the institution intends to take in order to overcome any perceived problems that have emerged during the review as well as those it plans to carry out to enhance the quality of its provision. Imp of SS Reports KSU Feb 11

5. Common Features of QA Agencies QA agencies typically set out a number of standards (sometimes referred to as sub standards) such as mission, resources, quality assurance etc and the self study has to specifically address each of these in addition to describing the institution. Imp of SS Reports KSU Feb 11

6. Differences between QA Agencies • At one extreme institutions are allowed to tell their own story, subject to the previous point about commenting on each standard • At the other extreme the agency closely controls the telling of the story; this often involves the answering of closed questions and the ticking of boxes • Which is best? It all depends. Imp of SS Reports KSU Feb 11

7. The length of the SSR Many agencies specify a maximum length of the SSR. Three main issues to consider • The need to tell a coherent story • The time it would take the panel to read it. • The need to avoid swamping the panel with excessive data – at the extreme there is trade off between data and information. Imp of SS Reports KSU Feb 11

8. How to provide additional information. A related issue is the SSR could not possibly contain all the information that the panel could conceivably need. Additional information can be provided • By way of appendices some or all of which might in electronic form • Files provided in the team’s resource room This can be a difficult balance to achieve (A remark made in pain following a recent review in which I was involved!) Imp of SS Reports KSU Feb 11

9. How to produce the SSR- 1 Most institutions do it ab initio, that is they basically start from scratch albeit drawing on data produced by the normal institutional processes and the last institutional SSR. But see the later point about a quality culture. Typically an institution-wide committee is set along with, depending on the nature of the institution, faculty or departmental committees along with others focussing on administrative processes. Imp of SS Reports KSU Feb 11

10. How to produce the SSR- 2 Those responsible for producing the SSR obviously need to consult widely throughout the institution as well as with external stakeholders. The student view is extremely important. The SSR should be made readily available throughout the institution. It does not auger well for the institution if the review team find that hardly anyone they meet in the institution contributed to the report or, even worse, know what is in it. But one can go too far as there are limits on the amount of time the institution can devote and too much discussion can confuse and alienate. Imp of SS Reports KSU Feb 11

11. The tone of the document It is vital to ensure that the SSR is not written by the marketing department! Excessive claims should be avoided and the institution should be aware that experienced reviewers will not believe that the institution has no problems or issues. An old joke (with a hint of reality) to produce a really good SSR the institution needs to identify two or three quite serious problems that it has faced and which it can show it is overcoming. Imp of SS Reports KSU Feb 11

12. But what if there are serious unresolved problems To rely on a cliché “Honesty is the best policy.” If you have a problem face up to it and do not allow the reviewers the possibility of discovering itself themselves. In an ideal world (but I am not suggesting we live in one) an institution that has a really serious problem should consult with the relevant agency and not wait until the institutional review to come around. Imp of SS Reports KSU Feb 11

13. Programme SSRs- 1 I have so far been concentrating on institutional SSRs, although many of the points apply equally to programme SSRs. Now turn more specifically to programme SSRs. Obviously programme SSRs have a more clearly defined set of issues to consider but they also have an additional function. Imp of SS Reports KSU Feb 11

14. Programme SSRs- 2 As stated earlier an institutional SSR should refer to the institution’s plans for the future that will only rarely include the option of tear everything up and start again. In contrast a programme SSR should include a consideration, based on the past history and a projection of future requirements, whether the programme should be torn up. Imp of SS Reports KSU Feb 11

15. Programme SSRs- 3 At the risk of oversimplifying a programme SSR has two elements – with the first of which informs the second. The first element is reporting on what the programme is achieved over the last, say, five years judged against contemporary relevant yard sticks and what was planned to be achieved as reflected in the last SSR or the original approval of the programme if this is the first SSR Imp of SS Reports KSU Feb 11

16. Programme SSRs- 4 The second element of the review should a rerun of the initial programme approval. In the light of current knowledge should we still be offering this programme or should it be closed or very substantially amended. While this should essentially be based on a current perspective (such as do we have the necessary resources, both in terms of quantity and quality) and an assessment of future needs the lessons of the past will also help in the assessment. Imp of SS Reports KSU Feb 11

17. SSRs and the Quality Culture - 1 Perhaps the simplest description of a quality culture is the state that exists when quality (and quality assurance) is built into everything that an institution does. Clearly an ideal that will never be achieved but one which it is worth striving towards Imp of SS Reports KSU Feb 11

18. SSRs and the Quality Culture - 2 Commented earlier that SSRs, especially institutional SSRs, are often started from scratch but increasingly this is not the case especially in the case of programme SSRs. The point being that the SSR is built on things that the institution should be doing in any event, irrespective of the requirement to produce an SSR. Imp of SS Reports KSU Feb 11

19. The link between annual programme monitoring and a five yearly SSR - 1 A well ordered institution should obviously keep its programmes under regular review and in particular monitor progress on annual basis. The monitoring should cover the numbers, the problems that have been identified and the steps taken to overcome them as well as the actions taken to improve quality. The annual monitoring report should also cover such things as discussions with external shareholders such as employers of the programme’s graduates. Imp of SS Reports KSU Feb 11

20. The link between annual programme monitoring and a five yearly SSR - 2 Thus the results of the annual monitoring will form an important part of the input of the programme SSR. However, there should be a major difference in that the programme SSR should be the occasion to “pull up the roots” which should not be done on an annual basis. The danger being that if the programme is fundamentally reviewed annually then very quickly it will be done on a mechanistic basis with little original thought. As ever, it is matter of balance. Imp of SS Reports KSU Feb 11

21. Quality Culture and the Institutional SSR Increasingly those responsible for the preparation of an institutional SSR will be able to draw on reports on the results of other activities that the institution would be producing even if it did not need to undertake an institutional SSR. While fundamental reviews of the institution’s role, mission etc will probably be triggered by external events rather than the need to produce an institutional SSR there is, I believe, still a need from to time to reflect on broader issues than would be appropriate to do an annual basis. Imp of SS Reports KSU Feb 11

22. My thanks for your attention – I look forward to our discussion. Richard Lewis rl@lewisuk.co.uk Imp of SS Reports KSU Feb 11