Summary of the discussion sessions at the Sixth Quality in Higher Education Seminar

1. Summary of the discussion sessions at theSixth Quality in Higher Education Seminar The End of Quality? 25–26 May 2001 Birmingham

2. The End of Quality? Three discussion themes: • Has external quality review had its day? • Has control of quality been usurped by the market and by information technology? • Does the development of mass education necessarily mean the end of quality?

3. External Quality Monitoring (EQM) EQM includes : • accreditation (of institutions and programmes); • institutional quality audit or assessment; • programme assessment or review; • external evaluation and comparison of standards; • research reviews; • ‘internal-external’.

4. Theme 1 Has external quality review had its day?

5. External quality monitoring leads to bureaucratisation and inflexibility? is amateurish? leads to ‘game playing’ and ‘performance’? short-term response not cultural changes? has a superficial impact on standards? has no real impact on student learning? is obsessed with accountability?

6. Bureaucracy • Any form of EQM would involve some level of ‘bureaucracy’. • Key issue is not the existence of a bureaucracy or of bureaucratic processes but the nature of the bureaucracy and its processes. • Bureaucracy must meets needs of external and internal stakeholders, not be self-perpetuating.

7. Quality monitoring bureaucracies Three main roles: • ensure integrity • initial • private; • act as a catalyst • improvement; • act as a conduit • qualitative • research • participatory not consumerist.

8. Quality process type legitimacy self-assessment improvement engagement longevity burden impact student feedback

9. Type and approach • Programme, subject or institutional level? • Less important than the purpose of the activity. • Dominant approach: self-assessment, peer review, statistical data. • Not necessarily seen as the best approach.

10. Self-assessment • Main value of EQM is the internal self-reflection. • But ‘two sets of books’. • Fear of revealing weaknesses.

11. Engagement • Engagement mediated by the perceived, short-term affect. • ‘Game playing’ and compliance. • ‘Performance’ to ensure maximum return. • Obscures the reality. • No surprise: ‘natural’ outcome of accountability-oriented processes. • Game playing taking up resources for very little real return.

12. Burden • However benign, imposed an unnecessary bureaucratic burden. • Time taken to prepare specific event-related documentation. • Evaluate on the basis of what institutions already produce.

13. Legitimacy • How far it is supported by academics. • Collaborative — not something being ‘done to’ an institution. • Help to improve outputs. • Some internal-external processes have legitimacy (e.g., external examining and departmental reviews), which external monitoring should not displace.

14. Improvement • Temporary impact. • EQM must interact with internal quality systems. • Changes in culture • slow • commitment. • Event or process: performance and game playing.

15. Longevity • Improvement potential decreases as process becomes more elaborate and routinised? • Emphasis shifts to procedural elements rather than innovative process. • Need constant reflection on and change in EQM, more trust and collaboration. • Periodic change in purposes and in the agencies themselves.

16. Longevity “ Without periodic change, there is the danger of ending up with a British-style, QAA-type, system: a rolling ‘juggernaut’, that is not sure what it is looking for, but which ensures compliance and minimizes innovation and risk-taking. British institutions continue to comply, even if the return on the investment is derisory, because of the fear of loss of income. ”

17. Efficiency • Doubts about the efficiency of most EQM. • Cost (external and internal) in no way reflect the value gained from the process. • Little engagement with the internal improvement process. • Periodic ‘events’ do not help inform change management. • EQM inhibits innovation through its conservative or rigid evaluation criteria.

18. Impact on learning • Extreme sceptical that EQM had any impact on programme quality or student learning. • No evidence of clear impact on learning • available research suggests that other factors outweigh the impact of EQM.. • Structure and organisation of EQM is not compatible with empowering staff and students to enhance the learning situation.

19. Student feedback • Important element of quality monitoring. • Link feedback to action and empowerment. • Over-evaluation of students, unlinked to action, leads to ‘questionnaire fatigue’ and cynicism by both students and staff . • Too much emphasis on a consumerist approach to feedback rather than a participative approach?

20. Theme 2 Has control of quality been usurped by the market and by information technology?

21. Market and IT HE business: infinite Internet providers? boundaried location for learning is obsolete? the market will arbitrate on quality? monitoring procedures stifle creativity? EQM is about restricting the number of providers?

22. Quality monitoring and gold standard • The market will not adequately monitor quality • at the very least need accreditation (e.g., US) • the market is not self-regulating • competition will not ensure the integrity of HE. • Underlying ‘gold standard’ notion of HE (international) • HE encapsulates a certain type of learning experience; a degree has a universal structure and integrity.

23. Impact of the market • The market has had limited impact despite predictions (US apart) • There will not be a higher education market that will determine quality. • Some aspects of a ‘market’ will impact on HE either to encourage improvement or inhibit it. • Despite their interconnections, distinguish between the impact of IT and of the ‘market’.

24. Public versus private • Important issue in some countries. • (Negative) impact on standards of fast growth in private sector. • Cherry-picking • public universities:basic ‘theoretical’ learning • private universities: lucrative (post-graduation) vocational skill development.

25. Pseudo-market conditions • Competition and efficiency • contingent funding, e.g., performance • lowering standards. • Diverse income streams. • Compliance • government agendas • distortion. • Diversity (and equivalence) • transformative potential for more students • possibly drives down standards.

26. Consumer choice • Internet • market challenge (but not distance) • international QM • integrate and enhance existing provision. • Student choice in learning style. • Marketable skills development.

27. Theme 3 Does the development of mass education necessarily mean the end of quality?

28. Mass higher education standards threatened by resource cuts and students doing more paid work? student outcomes are different not worse? employer demands have changed? grade inflation (especially at the bottom end)? external pressures to increase pass rates? EQM is concealing the drop in standards and the reduction in the unit of resource?

29. Mass higher education • increase the general level of education, skills and abilities; • recruitment of less well qualified students; • necessitating greater value added to avoid decline in ‘standards’; • most governments are not prepared to fully fund expansion of higher education; • resulting in access and equal opportunity issues.

30. Value-added “ It is unfortunate that so little advance has been made in developing value-added approaches or measures. Although it might be difficult to measure student progress in any quantifiable way one suspects that there was no political will to develop value-added, not least because it might turn conventional league tables upside-down. ”

31. Student experience • Massification gives rise to concerns about the quality of the student experience. • Lowering of service standards to students • Fewer resources, less staff time. • Less formative assessments. • More paid work, less peer interaction. • Different rather than worse experience. • More emphasis on explicit attribute development.

32. Standards • Little support for the notion that standards in HE are declining. • HE has changed and that current and past standards cannot be compared. • Value-added is improving. • New era in HE, which needs new methods of evaluating standards that provide for continuous assessment and allow for assessment of skill development

33. Quality as agent for change • EQM as agent of change or vehicle for government policy. • Teaching ability is now expected: staff development, to enhance teaching ability. • Quality processes as a means to change curricula and priorities: • responsiveness to external factors. • Biggest problem of quality reviews is legitimation of declining resources.

34. A Quality Manifesto “ Academics and students of the world unite and reclaim the quality agenda….. ”

35. Thank you http://www.uce.ac.uk/crq http://www.qualityresearchinternational.com