Judy Morton, Ph.D. Vice President Quality Integration & Improvement Swedish Medical Center Seattle, WA (206) 386-602

1. Measurement Framework: Malcolm Baldrige Education and Healthcare Criteria for Performance ExcellenceMaternal and Child Health Leadership Training ConferenceApril 19, 2004University of WashingtonSeattle, WA Judy Morton, Ph.D. Vice President Quality Integration & Improvement Swedish Medical Center Seattle, WA (206) 386-6027 judy.morton@swedish.org

3. What Is Included in the Performance Excellence Model? • The criteria...Key elements to optimize organizational performance • A set of values

4. Malcolm Baldrige PerformanceExcellence Framework Organizational Profile: Environment, Relationships, and Challenges Strategic Planning Faculty and Staff Focus 85 85 Leadership Organizational Performance Results 120 450 • Customer/ • Market Focus • Patients • Students Process Management 85 85 Measurement, Analysis, and Knowledge Management 90 2004 Criteria for Performance Excellence Baldrige National Quality Program

5. Baldrige Criteria for Performance Excellence • Seven category framework…different weights, with an emphasis on results • Applies to various sectors/ organization types • Non prescriptive and adaptable • Updated regularly • As assessment tool

6. Core Values Education Healthcare • Visionary leadership • Organizational and personal learning • Management by fact • Focus on results and creating value • Systems perspective • Agility • Focus on the future • Managing for innovation • Patient centered excellence • Social responsibility and community health • Valuing staff and partners • Learning centered education • Social responsibility • Valuing faculty, staff and partners

7. Who Was Malcolm Baldrige? • 26th Secretary of Commerce • Excellent manager • Commerce Department • Scovill, Inc. • “Colorful and beloved”

8. Why Do We Care? • National Quality Award named after him to: • Promote America’s competitiveness and performance • Facilitate fundamental changes in US business practices • Recognize excellence • Establish guidelines for self-assessment of management systems • Facilitate dissemination of practices leading to world class performance • The model, when rigorously applied, produces documented world class results • May have applicability to MCH

9. Baldrige Recipients: Education • Chugah School District (pre-school to post secondary education in Alaska) • Pearl River School District (K  12 in New York) • University of Wisconsin - Stout • Community Consolidated School District 15 (19 elementary and junior high schools in Illinois

10. Baldrige Recipients: Health Care • SSM Healthcare - St. Louis, Missouri • Baptist Hospital - Pensacola, Florida • St. Luke’s Medical Center - Kansas City, Kansas

12. Malcolm Baldrige PerformanceExcellence Framework Organizational Profile: Environment, Relationships, and Challenges Strategic Planning Faculty and Staff Focus 85 85 Leadership Organizational Performance Results 120 450 • Customer/ • Market Focus • Patients • Students Process Management 85 85 Measurement, Analysis, and Knowledge Management 90 2004 Criteria for Performance Excellence Baldrige National Quality Program

13. Results: Education and Healthcare 450 of 1000 possible points Education Student learning Student- and stakeholder-focused Budgetary, financial and market Faculty and staff Organizational effectiveness Governance and social responsibility Healthcare Healthcare Patient- and other customer-focused Financial and market Staff and work system Same Same

14. Key Considerations Performance Measurement • Reliable, valid measures • Performance over time • Performance compared to others • Competitors • Average • Best in industry or class • Segmentation

15. Sample Measures: Student Learning (Education) • Student retention rates from year to year • Percent of students meeting/exceeding grade standards • Significant differences in student performance by demographic group • Percent of fully licensed graduates within specified time periods • Student grades with and without supplemental instruction • National board results, e.g., passage rates, average scores • Demonstration of leadership competencies • Percent of training programs using leadership competencies • Percent of graduates who obtained masters/doctoral degrees and/or board certification • Percent of graduates serving targeted populations

16. Sample Measures: Healthcare • Risk adjusted mortality rates (overall, by service line) • Ventilator-associated pneumonia rates • Nosocomial infection rates • Medical staff clinical indicator index • Pressure ulcer prevalence • Surgical infection rates • Performance levels: national teaching hospitals comparative group of obstetrical and perinatal indicators (e.g., C-section rates) • Medication event rate • Above measures segmented by gender, ethnicity, etc.

17. Sample Measures: Student/Patient Results Student Results • Student satisfaction with instruction: current students and past students • Over time • Compared to “competitor” and/or Baldrige recipient • Differences among subgroups of students, e.g., gender, grade level • Student confidence levels • Student accidents and/or perception of safety results • Ratio of number of students applied/number selected (selectivity) Patient Results • Patient satisfaction levels: inpatients, outpatients, ED, etc. • Over time • Compared to others • By service line or site • Complaint/compliment ratios

18. Sample Measures: Budgetary, Financial and Market Results • Operating cost per student • State and/or federal revenue per year • Personnel or instruction expenditures as a percent of budget • Budget performance • Student loan default rate • External ratings of market excellence • Market share • Public and private funding for research training

19. Faculty publications Student/faculty ratios New faculty/staff satisfaction with orientation Professional development expenditures/person Faculty and staff turnover and/or vacancy rates All faculty and staff First year Percent of faculty with doctorates Percent of minority faculty/ staff compared to community served Faculty/staff satisfaction results Percent of nationally certified teachers Percent of certified substitute teachers Staff/faculty safety and worker’s compensation claims Salary survey results Labor grievance rates Sample Measures: Faculty and Staff Results

20. Sample Measures: Organizational Effectiveness • Employer ratings of preparedness of graduates • Percent of students who graduate within specific time period • Student job placement rates • Percent of graduates in healthcare leadership positions within X years of graduation • Cycle time reduction: key processes, e.g. technology service time, purchase order turnaround time • Network down time • User satisfaction: key support services, e.g., IT, HR, library services • Percent of students who access the internet • Percent of graduates attending post-secondary institutions • External awards • Percent of alumni who say they would go through the program again

21. Sample Measures: Governance and Social Responsibility • Percent of students, faculty and/or staff providing services to the community • Hours of service per person: community outreach services • Accreditation or external survey results • United Way contributions • External audit recommendations • Public complaints • Percent of faculty/staff providing technical assistance to the community • Percent of faculty, students, and/or graduates participating in advocacy • Percent of graduates working in underserved communities

23. Malcolm Baldrige Quality AwardInformation and Tools • Telephone: (301) 975-2036 • Email: nqp@nist.gov • Web site: www.baldrige.nist.com • Copies of Criteria • Telephone: (800) 248-1946 • Web site: www.asq.org

24. Other Ways to Learn More At national or state levels... • Attend workshops • Become an examiner • Complete an organizational self-assessment • Apply for the award

25. Rising to the challenge… identifying key measures Citigroup, 2004

26. Considerations: Measure Selection... • MCH national training plan • Assure a workforce capable of meeting MCH population needs • Prepare/support a diverse workforce…culturally competent and family centered • Improve practice through interdisciplinary training • Develop effective MCH leaders • Develop new knowledge (training, policy, outcomes) • Balance of outcome and process measures • Key stakeholders and customers, e.g., patients, community, students/trainees, faculty/staff • Shared and unique measures • Performance over time and compared to others • Balance between multiple, sometimes competing systems, e.g., training program vs. academic goals