Haroutune Armenian Professor in Residence, UCLA President Emeritus, American University of Armenia

1. Challenges of Quality Assurance and Accreditation in Higher Education.  Are health care models relevant? Haroutune Armenian Professor in Residence, UCLA President Emeritus, American University of Armenia Professor Emeritus, Johns Hopkins University

2. OUTLINE 1. Health care quality and Donabedian’s Structure – Process – Outcome 2. Evolution of Educational Accreditation 3. Parallels between health care and educational accreditation 4. Potential applications from health care to educational quality assurance 5. Issues for a successful Accreditation

3. Quality Assurance in the Delivery of Health Care

4. Quality Assurance • The historical terminology used in health care to describe a set of activities aimed at assessing, evaluating, and maintaining a desirable level of performance. • During comparative analysis across geographical regions, the localization of quality has to be taken into account since the same type of performance can be evaluated differently across geographical regions placed within their cultural context.

5. Donabedian model ofStructure – Process – Outcome for quality

6. STRUCTURE PROCESS OUTCOME

7. Structure/Process/Outcome • Structure—Does a structure exist to implement the health care intervention (program) and what are its characteristics? • Process—Is the process to implement the health care intervention (program) working? • Outcome—What effect has the intervention had on the outcome(s) of interest?

9. PROCESS OUTCOME

12. The Range of Health Outcomes • Mortality—all cause or cause-specific • Morbidity • Disease specific indicators • General indicators—clinic use or hospitalization, medication use • Quality of life • General • Disease-specific • Costs

13. Efficacy, Effectiveness, Efficiency • Efficacy: How well does the intervention work in a trial setting? • Effectiveness: How well does the intervention work in the “real world”? • Efficiency: How much does the intervention cost? What is the cost-benefit ratio?

14. Evaluation • Efficacy – Works • Effectiveness – Application • Efficiency – Resources

15. Health Care Outcome CLASSIC HEALTH SERVICES RESEARCH INTO EFFECTIVENESS

16. Environmental and other factors Health Care Outcome CLASSIC HEALTH SERVICES RESEARCH INTO EFFECTIVENESS

17. INDUSTRIAL vs. HEALTH CARE models for Quality - 1 Health care has a more complex set of responsibilities toward the individual and society. (Donabedian) 1. Need to act on behalf of the patients and consumers 2. Importance of engaging individual consumers and society in the management process.

18. INDUSTRIAL vs. HEALTH CARE models for Quality - 2 • 3. Responsibility for social welfare in addition to individual welfare. • 4. Improvements in quality cost more (diminishing returns in health following further increases in care). • 5. Tremendous variability that health care professionals have to deal with. No standardized solutions. Uncertainty is pervasive.

19. Evolution of Educational Accreditation

20. The Basics • Accreditation: regional, national, specialized • The Regionals: NEASC, MSACHE, SACS, HLC/NCA, WASC (Senior ), WASC (ACCJC), NWCCU • The numbers: ~ 50 staff work with > 3000 institutions; teams vary in size by region from 4 – 15 • The challenge: evaluate every institution (from 30 – 48,000 students) in 4 days (2 on campus)

21. Accreditation and Assessment • USDE Recognition: 9 areas for standards req’d, student academic achievement adopted in 1984, moved to #1 in 1998 • Accrediting Standards: started as institutional effectiveness by SACS in 1980’s, then by WASC and MSA. • Shift from institutional to educational effectiveness • Commission on the Future of Higher Education

22. Themes of Accreditation Reform • The shift in accreditation from an episodic ‘add-on’ to a value adding process • The shift in definition of quality from inputs and processes to learning outcomes • A shift from a single comprehensive visit to a variety of new visit processes • A redefinition of accreditation from a conservator of traditional values to an agent for change

23. Key Issues From Accountability Debates • Does the federal government have the authority to set standards for student learning? • What is good enough? • Graduation/job placement rates • Licensure pass rates • Learning outcomes • Who decides? USDE, accreditors, institutions? • What is the unit of analysis? Program or institution?

24. What Will the World Look Like For 2006 Graduates in 20 Years? • 1 in 4 workers have been with their current employer <1 year; I in 2 < 5 yrs. • Today’s workers will have 10- 14 jobs by their 38th birthday • How many majors or concentrations did not exist 10 years ago? 10 years from now? • Today’s 21 yr olds have watched 20,000 hours of TV, played 10,000 hours of video games, sent/received over 250,000 email or text messages

25. As we enter into the 21st Century, what are the most important learning competencies our students/graduates need?

26. Parallels between health care and educational accreditation Armenian, HK. et al Quality Assurance Parallels in Health Care Evaluation and Educational Assessment: The American University of Armenia Experience. Education for Health Journal, Epub. July 2009.

27. Parallel Development with a Lag Time • confluence of approaches between the evaluation of quality of health care and assessment of educational effectiveness • 1970s emergence of evaluation systems in health care that have evolved toward assessment of outcome as an indicator of quality • similar evolution toward an assessment of impact has occurred in the evaluation of educational effectiveness

28. Parallel Development 1 • The measurement of performance for both health care services delivery and of institutions delivering educational programs are remarkably comparable. • In both instances, an understanding of the environment, the process of delivering the services (be that health care or education) and the impact those activities and contexts have upon the recipient of the services follow a similar methodology.

29. Parallel Development 2 • The responsibility of the organizations providing the care is expected to encompass the application of the evidence-based methods to maximize the potential for either improving health or empowering students with new knowledge and skills. • To a large extent, it is the responsibility of the organization to build a continuous self-improvement methodology whereby not only the evaluation of the goodness of the services but also continuous learning of the organization itself takes place.

30. E. Codman, a surgeon in early 1900s • Massachusetts General Hospital, put forth the need of peer and public evaluation of physicians. He was a visionary in the development of health care systems. • “…In various manufacturing businesses I imagine that it is not difficult to render an exact account of the product of the factory. So many dozen tin cans, cakes of soap, toothpicks, … are readily figured up. With educational institutions and hospitals the problem is very different. The statement of the number of patients treated or of students graduated gives but a fraction of the products of such institutions. What, then, are the products of a large hospital, whether in the forms of healed wounds, healthy babies, faithful nurses, promising young surgeons and physicians, or in the more abstract forms of original ideas on pathology or treatment, model methods of administration, or such intangible things as enthusiasm and ideals?”

31. Accountability of Organizations 1 • However, even professional organizations are in need of structured accountability. One mechanism for such accountability is accreditation of the organizations or the licensure of the direct care providers. Again, both health care organizations and educational institutions share this similarity in need.

32. Accountability of Organizations 2 • Accountability is at the heart of demonstrating that the relationship between structures, processes, and the desirability of the outcomes are established in a systematic and verifiable way. • There are structural requirements for any organization that would receive accreditation or licensure, mainly that the setting is conducive to the delivery of the specific services, that those who deliver the services are qualified, and that the interaction between those providing services and the recipient is not conducive to harm either through the recipient or the providers.

33. From Health Services to Education Common denominators that characterize institutions of good quality medical care: • The presence within the institution of good quality medical records that document accurately patient transactions and evaluations. • Teaching hospitals have always fared better than non teaching hospitals. • Surveillance and monitoring of adverse outcomes and events. • Evaluation in health services is very dependent on the comparative method.

34. Example: Bahrain • Office of Professional Standards and Systems Analysis: • Salmaniyah Quality Assurance Committee • Medical Records • Health Center Continuing Education, System Change and Quality Review Process • Patient Satisfaction Surveys

35. Example: American University of Beirut • Long Standing JCAH accreditation reviews • Chief of Medical Staff position – “ the conscience of the institution” • Data system to review and monitor statistics • Quality Assurance office • New York Board of Education

36. The Soviet Model • 1960s “The system has got the right anatomy but little physiology.” • 2000s - a lack of physiology but the anatomy is also structurally not able to stand on its own. We need to relate processes of care to their outcomes. • The dignity of having a job has to be transformed into the dignity of achieving healthy outcomes.

37. Example: Nork Marash Medical Center • Partnership with the AUA College of Health Sciences • Checklist and baseline survey. • Identify improvements needed. • Develop a plan of implementation. • A repeat survey every 1-2 years. • Components of System Change, Education and Evaluation.

38. Example: American University of Armenia US Accreditaion • Eligibility • Candidacy • Accreditation

39. ACCREDITATION

40. As a University What do we Expect from the WASC Educational Effectiveness Review? • Continuing Peer Review – quality assurance • Guidance as to where we are and where we can be? • A process that assists us in institution building for the future. • Making the University more efficient. • Some closure of a five year process.

41. September 2004 Letter to WASC • Commission concerns addressed • Academic freedom statement, Academic Standards Committee, classroom space, BOD, Library, faculty by-laws, Alumni-career office, student council, tuition revenue etc. • Strategic Planning • Scholarship and loan program • Educational effectiveness measurement • Fiscal controls and viability

42. WASC Activities • Steering Committee • George Jakab, Don Fuller, Penny Brunner, Lucig Danielian • Self Study and Strategic Planning process: • Academic Units • University-wide • Meetings with the WASC Review Committee in March and May • Judie Wexler’s visit

43. WASC Recommendations 2005For Educational Effectiveness • Are students learning outcomes clearly stated and in what way can the institution document whether these outcomes are being achieved? • Are there results from investigations that are based on direct evidence of student work? • What processes are in place to insure regular and periodic review? • Are course syllabi aligned with course outcomes? • In what ways are the faculty engaged in this process?

44. Preparation for the 2006 Review • October 2005 - 3 working groups • Management Information System • Impact of the University • Learning Effectiveness • Regular meetings and preparatory process • External Review by Dr. Artin Aslanian • Meetings with WASC

45. Concluding Perspectives • In 1997: Defining a role for the AUA: Choice, Entrepreneurship, Quality, Technology • In 2007: Excellence through Relevance to the broader Community • Quality: achieving some established standards (mostly structure and process) • Excellence: the moving target of the best (mostly outcomes?)

46. OUTLINE 1. Health care quality and Donabedian’s Structure – Process – Outcome 2. Evolution of Educational Accreditation 3. Parallels between health care and educational accreditation 4. Potential applications from health care to educational quality assurance 5. Issues for a successful Accreditation