A New Approach in University Evaluation: the Balanced Scorecard

1. A New Approach in University Evaluation: the Balanced Scorecard Angela Maria D’Uggento, Massimo Iaquinta,Vito Ricci Università degli Studi di Bari Direzione Analisi Statistica – Area Studi, ricerche e programmazione Staff di Rettorato Statistical modelling for University Evaluation: an International Overview Università di Foggia- Mattinata (FG), 5th-6th September 2008

2. A New Approach in University Evaluation: the Balanced Scorecard Balancedscorecard: a definition The balanced scorecard (BSC) is a strategic planning and management system used to align business activities to the vision and strategy of the organization and to monitor organization performance against strategic goals. It embodies a set of measures that gives top managers a quick but exhaustive vision of the business; in fact, it complements the financial measures with operational measures on: 1) customer satisfaction; 2) internal processes 3) organizational innovation; 4) improvement activities (Morard et Stancu, 2005). It was created by Kaplan and Norton as a performance measurement framework that added strategic non-financial measures to traditional financial ones (Kaplan, Norton 1992).

3. A New Approach in University Evaluation: the Balanced Scorecard Balancedscorecard The BSC has evolved from its early use to a full strategic planning and management system (Kaplan, Norton 1996). Born in the business managerial field, in the last decades it has drawn the attention of public administrators, becoming a useful tool to carry on evaluation during years. Some foreignUniversitieshaveadopted the BSC: Edinburgh (UK), San Diego, California (USA), Bond (Australia). AnyItalianUniversityhasnot a BSC butit’s currentlyusedeitherby private companies or by Public Administration (Health Care, LocalGovernment, etc).

4. A New Approach in University Evaluation: the Balanced Scorecard Balancedscorecard: the 4 traditionalPerspectives • Financial: itconcernswitheconomic and financialaspects; • Stakeholders:thisperspectiveallows the Organizationtofitoutcomemetricsto “customers” needs and expectations; • Internalprocess: it’s the perspectiveofoperationalprocesses; d) Learning and Growth: itincludesemployee training, empowerment, informative systems.

5. A New Approach in University Evaluation: the Balanced Scorecard BSC: traditionalperspectives’ relations

6. A New Approach in University Evaluation: the Balanced Scorecard Balancedscorecard: mainsteps • Specificationofstrategicobjectives and goals • Definitionof a StrategicalMaptohighlightcause and effectrelationshipsamongstrategicobjectives • Selectionof the KPA (key performance actions) and the KPI (key performance indicators) • Implementationof the BSC dashboardformonitoring and evaluation

7. A New Approach in University Evaluation: the Balanced Scorecard The Balancedscorecardof the Universityof Edinburgh/1 Fourperspectives and performance indicators

8. A New Approach in University Evaluation: the Balanced Scorecard The Balancedscorecardof the Universityof Edinburgh/2 Fourperspectives and performance indicators

9. The Balancedscorecardof the Universityof Edinburgh/3 Specificationof the single indicator (1)

10. The Balancedscorecardof the Universityof Edinburgh/4 Specificationof the single indicator (2)

11. A New Approach in University Evaluation: the Balanced Scorecard A BSC foranItalianUniversity: the revisedPerspectives (Del Sordo et al, 2007) d • Students (Academic Courses) (needs and expectations) • Community (Research) • Learning and Growth (employee training, empowerment, informative systems.) • Internal processes (operational processes) • Economic and financial (efficency & efficacy) Stakeholders Learning & Growth Internal Processes Financial

12. A New Approach in University Evaluation: the Balanced Scorecard A BSC forUniversity: Ourproposal • IntroductionofMiURfiveGuidelines and IndicatorsaboutUniversity Planning 2007-2009 (MiUR, DM n.362/07 e DM n. 506/07) into the BSC standard frame; 2) Translation of the six guidelines of Bari University Planning into the four BSC traditional dimensions.

13. A New Approach in University Evaluation: the Balanced Scorecard A BSC for University: Our proposal The perspective Stakeholders would be divided into 4 sections (Figure 1): • Academic courses, • Development of scientific research, • Relationship with territory, • Internationalization programs so representing all the possible “players”: students and their families, researchers, local, domestic and foreign Institutions and firms.

14. A New Approach in University Evaluation: the Balanced Scorecard Figure 1 - BSC Perspectives and Guidelines of University planning

15. A New Approach in University Evaluation: the Balanced Scorecard A BSC forUniversity: MiURGuidelines & EvaluationFields (from DM 506/2007) • Academiccourses(3 indicators) • ScientificResearch(5 indicators) • ServicestoStudents and Researchers(5 indicators) • InternationalizationProgrammes(4 indicators) • Teaching and non-teaching staff policies(4 indicators) • Relationshipswithterritory (Bari Plan 2007-2009)

16. A New Approach in University Evaluation: the Balanced Scorecard A BSC for Bari University Strategic objectives, Initiatives/actions & targets Key Performance Indicators UniversityPlan 2007-2009 21 MiURindicators + 21 Performance indicators

17. A New Approach in University Evaluation: the Balanced Scorecard A BSC for University: some performance indicators of Bari University

18. A New Approach in University Evaluation: the Balanced Scorecard A BSC for University: work in progress If implemented in Italian Universities, BSC could represent: • a standardized model and as a reporting tool for decision-makers and stakeholders; • a framework to measure the multidimensional performance of the University (intellectual & relational capital), whose core business is producing and offering knowledge intensive services and to maximize students’ satisfaction; • a modern and integrated approach, particularly appreciated by the University management which, sometimes, hardly succeeds in monitoring, at the same time, several variables which are usually untied; • an instrument to rationalize data collection and to minimize information overload

19. A New Approach in University Evaluation: the Balanced Scorecard A BSC for University: work in progress There is no doubt that one criticism of the BSC is its construction. Sometimes, the conceptualization is very hard to apply in a university environment, but a modeling of the system is required as a starting point (how many Italian universities have “mapped” their internal processes ?). As to the methodological approach, in recent years the research directions include the structural equation modeling, encompassing Partial Least Squares models, and the multivariate approach, in addition to the traditional statistical techniques.

20. A New Approach in University Evaluation: the Balanced Scorecard A BSC for University: further developments • the introduction of a BSC concerning the whole Italian Universities System (using mean or median of single Universities’ indicators with also standard deviation). It could be a useful tool for the Italian Ministry of Higher Education; • a standardized instrument of benchmarking among Italian Universities; • after measuring and managing performances, an instrument to evaluate them, either at the whole University level or at Faculty or Department level; • to apply a structural equation modeling in a rationalization tentative of BSC (Morard & Stancu, 2005)