Agenda Setting and Improvement Monitoring in a University Department

1. Agenda Setting and Improvement Monitoringin a University Department Stuart Umpleby Research Program in Social and Organizational Learning The George Washington University Washington, DC USA Email: umpleby@gwu.edu Igor Dubina School of Economics and Management Altai State University Barnaul, Russia Email: din@gwu.edu Deming Conference New York City, February 2006

2. The research was conducted in the Department of Management Science George Washington University Igor Dubina was a visiting scholar at GWU in 2004-2005 under the Junior Faculty Development Program(JFDP) of the U.S. Department of State

3. Quality Improvement Priority Matrix(QIPM) • 1995, 1996 Baldrige Award Conferences • A method for achieving data-driven decision-making • QIPM is a way of focusing management attention on high priority tasks. It can be seen as an alternative to control charts • Features of an organization (or product or service) are rated on two scales – importance and performance • Scales range from 1 to 9 • The measures that result are averaged Importance (I),Performance (P), and Importance/ Performance Ratio (IPR)

4. QIPM

5. Data was collected from members of the GWU Department of Management Science in 2001, 2002, 2003, and 2005 They evaluated features of the Department(a total of 52 features): • Funds to support research • Salaries • Coordination with other depts. • Computer labs • Classroom facilities • Classroom scheduling • Office space for faculty • Travel support • Dept. and School websites • Library book and journal collection • Office security • English skills of students • Course evaluations • Teaching assistants • Faculty annual reports • Conference room and other space • Computer hardware and software • Course catalogue • Copiers • Secretarial support • Dept. strategic plan

6. The most stable high importance features(always in the first 15)from 2001 to 2005

7. The most stable low importance features(always in the last 15)from 2001 to 2005

8. The most stable low Performance features (always in the last 15) from 2001 to 2005

9. The most stable high Performance features (always in the first 15) from 2001 to 2005

10. The features always in the SE quadrant from 2001 to 2005

11. A classical approach: features in the SE quadrant are considered to have a high priority Visual analysis of QIPM does not discriminate features’ priorities sufficiently • From 1/3 to 1/2 of all features routinely fall into the SE quadrant (e.g., 19 of 51 features in 2001, 17 of 52 in 2002, 23 of 52 in 2003, and 26 of 52 in 2005 • The “border effect” • The problem of automatic clustering of factors by their priorities

12. Using average Importance and Performance as a midpoint rather than the scale midpoint

13. Clustering features by the IPR interval Cluster 0 (urgent) – IPR>2 Cluster 1 (high priority) – [1.5 – 2] Cluster 2 (medium priority) – [1.25 – 1.5) Cluster 3 (low priority) – IPR<1.25 rIP = 0.96 (0), 0.88 (1), 0.85 (2), 0.90 (3) rIP = 0.18 (unclustered) A way to automatically cluster features with different priorities is to choose intervals that create clusters with the highest correlation coefficient

14. An approach to automatically cluster features with different priorities P=a0+a1I+b1C1+b2C2+b3C3 , r2 P– Performance I – Importance C1, C2,and C3 – dummy variables corresponding to clusters (These variables have values 1 or 0 depending on whether a point is or is not in the corresponding cluster: 1, 2, or 3) Coefficientsb1, b2, b3represent the increased performance for each cluster compared with the cluster 0 r2– coefficient of determination The higher r2 is in this regression equation, the more precise the clustering

15. Clusters with different priorities(2005 г., r2 = 0.90 )

16. A simplified approach P=a0+a1I+a2X P– Performance I – Importance X – a dummy variable corresponding to the number of the cluster. It may have values 0, 1, 2, or 3 if a point falls into the corresponding cluster. The coefficient a2 represents the average shift in performance between clusters. 2005: r2 = 0.89

17. An integrated approach The parameters for automatically clustering features with different priorities • Number of clusters • IPR intervals • Number of features in clusters • Correlation of features in clusters • The coefficient of determination • Average shift in performance between clusters

18. Analysis of year-to-yeardynamics dIPR = IPRt2 – IPRt1 represents the direction of movement (becoming more urgent or less urgent) represents the amount of movement 3 clusters with different levels and directions of change: • DI >= DItи dIPR>0 (regress and greater urgency) • DI >= DItи dIPR<0 (progress and less urgency) • DI < DIt (change is not significant)

19. Multiyear analysis of feature dynamics dIPR = abs(IPRt1 – IPRt2) + abs(IPRt2 – IPRt3) + abs(IPRt3 – IPRt4) reflects changes between clusters with different priorities represents the amount of movement The more important movement was between clusters, even if the distance moved was not as great. IPR significantly changes when a feature moves in a perpendicular direction (from cluster to cluster). Movement between clusters means a change in priority. Therefore, the indicator dIPR is more important for analyzing changes in priorities.

20. QIPM • Is easy to understand • Is efficient in terms of time and resources • Provides enough precision for monitoring changes in priorities and performance • Is based on subjective data, so can be used to extend process improvement methods beyond manufacturing into service-oriented activities

21. Presented by Stuart Umpleby at the • Deming Conference • Fordham University • New York City • February 13-14, 2006