Błażej Feret The Main Libray of theTechnical University of Łódź, Poland

1. Library performance indicators does it really make sense to measure them? Data collection and analysis of library performance a case study for Polish research libraries Błażej Feret The Main Libray of theTechnical University of Łódź, Poland blazej.feret@sunlib.p.lodz.pl Marzena Marcinek Cracow University of Technology Library, Poland marcinek@biblos.pk.edu.pl CASLIN Seminar, June 2006

2. Ackowledgements Special thanks go for Mrs Lidia Derfert-Wolf, a member of the Task Group for Standardisation for Polish Research Libraries, for her kind assistance and advice.

3. Plan of the workshop 1. Introduction15 minutes 2. A few general remarks on library quality and performance measurement Quality criteria from different perspectives 10 minutes 3. A Common Project of Polish Research Libraries on Comparable Measures (incl. examples) 40 minutes Czech realities 20 minutes 4. Standardised terminology and descriptions of library performance indicators based on the ISO 11620 Standard Information and documentation – Library performance indicators20 minutes • Conclusions15 minutes

4. A few general remarks on library quality and performance indicators

5. Quality: fitness for purpose fitness for use conformity to requirements absence of defects …… Quality of libraries (ISO 11620): totality of features and characteristics of a product or services that bear on the library's ability to satisfy stated or implied needs

6. Quality of research libraries from the perspective of: Funding bodies Users Librarians Please consider all these perspectives

7. Model of measuring quality of libraries • inputs- the raw data such as finance, collection, equipment, users and staff, space, seats • outputs - the work done, i.e. circulation, cataloguing, reference services, preservation, interlibrary lending, facilities usage and e-sources searches • outcomes – user satisfaction and the impact of library services on users at the local institution and society • quantitative data • qualitative data • from the perspective of: • users • funding bodies • librarians group work

8. Library quality assessment- what is required? • set of quality criteria library’s goals and objectives • set of performance indicators • national library statistics system • standardisation on local, national and international level but ...

9. “The level of quality cannot be defined once and for all, since both the criteria and evaluation methods, as well as the assessment of the results achieved, may change. This is caused by various factors, for example technological, political, economic, as well as the ones connected with the community in which and for whom information services work. It is crucial that these varying criteria and methods, as well as the dissimilarity in the level of the quality achieved by a given country, are taken into account when aiming at adopting the international standards on the quality of information work. Countries differ in living standards they achieve, in the level of education, and the way they introduce innovations. Setting the goal is of fundamental importance, since its accomplishment or failure to reach it may be treated as a basis for quality assessment”. W. Pindlowa

10. Activities conducted by the Group for Standardisation for Polish Research Libraries, results and plans for the future CASLIN Seminar, June 2006

11. The Group for Standardisation for Polish Research Libraries coordinated by the Adam Mickiewicz University in Poznań • Lidia Derfert-Wolf (ATR, Bydgoszcz) • Ewa Dobrzyńska-Lankosz (AGH, Krakow) • Mirosław Górny (Adam Mickiewicz University, Poznan) • Elżbieta Górska (Warsaw Public Library) • Marek Górski (Cracow University of Technology) • Artur Jazdon (Adam Mickiewicz University, Poznan) • Dariusz Pawelec (Silesian University, Katowice) • Anna Sokołowska-Gogut (Academy of Economics, Krakow) • Teresa Wildhardt (Pedagogical University, Krakow)

12. Objectives • to gather libraries' statistical data for a computer database • to select a set of performance indicators and standards for library performance (quantity, quality and effectiveness) • to conduct a comparative research • to prepare and publish yearly reports • to define methods for the assessment of Polish research libraries

13. Project for the Analysis of Polish Research Libraries

14. Some facts as of May 2006 • Libraries registered – 59 • 54 academic libraries (42 state-owned and 11 non state-owned) • 3 public libraries • 2 special libraries • Questionnaires for 2004 completed 31 libraries • Questionnaires for 2003 completed 29 libraries • Questionnaires for 2002 completed 17 libraries

15. Softwarefor the collection and analysis of data - general requirements • on-line access to the questionnaire (submission, modification) • selected performance indicators automatically calculated and presented • automatic control and verification of the accuracy of submitted data • access to analysing functions for individual libraries • possibility to conduct a multi-aspect comparative analysis of selected data and performance indicators

16. Web-based Application • for: • General users • Library directors • Analysts • Administrators

17. General user • General information about the Project • Instructions • Registration • Rules for filling up questionnaires • Data analyses • Questionnaires (patterns in doc. format) • Example results • Useful links

18. http://ssk2.bu.amu.edu.pl/standaryzacja/index.htm General information Instructions Questionnaires Results handouts 1 Useful links

20. Library Director • Filling up questionnaires; • Analysis of the own library data sorted by years, question categories or indicators; • Access to calculated automatically indicators • Comparison of the own library data with average results of other libraries in the same category or in the country (data published in the form of tables). • Establishment and modification of access rights for the library staff

21. Login

22. For directors of registered libraries Users Questionnaires Analyses Send e-mail Change password Logout

23. handouts 2 Questionnaire • 48 questions of various types • refer to easily accessible or computable data (e.g. size of collection, number of users etc.) • closed questions about the services offered (e.g. on-line reservation: Yes/No) • 88 performance indicators • 19 calculated by librarians • 69 calculated automatically

24. Questionnaire • addresses all the elements of a library system, its environment, library processes and services • divided into chapters: • Staff • Collection • Budget • Infrastructure • Circulation • Information services • Didactics • Publications and data bases created by the library • Library cooperation • Organisation of library events • Professional activity of library staff • ,

25. Examples of performance indicators required to complete the questionnaire • library’stotal expenditures • expenditures for library materials/books • ratio of library budget to the budget of its parent university • time required for the technical processing of a document • collection on the computer system as a % of the whole collection of the library • percent of catalogue descriptions acquired from outside resources

26. Examples of performance indicators calculated automatically • library expenditure per student/user • expenditures for library materials/books per student/user • library registered usersas a percent of potental users • space of the library per user • collection on the computer system as a % of the whole collection of the library • number of user training hours per one staff member

27. Survey form For university libraries No of libraries in the university library and information system (incl. the main library) data concern main library university library and information system Staff commentary No of library staff categories education age

28. Collection Acquisition

29. Budget

30. Infrastructure Circulation

32. Standard lending period Information services Electronic sources usage Didactics Library’s own publications and databases

33. Library’s own publications and databases Interlibrary cooperation, staff professional activities

34. Selected performance indicators

35. Once the questionnaire is validated the indicators are calcullated automaticly Indicators calculated automatically Admin can add other indicators based on the data gathered - no action by director required

36. Adding and modifying users

37. statistical data values Director can analyse performance of his/her own library for selected years

38. categories for the analysis Tyle kategorii ile pytan i wskaznikow wpisanych i automatycznych Number of categories = number of data and indicators (entered and calculated automatically)

39. Total library space per student No of loans per registered user No of loans per staff membber

40. No of years analysed: 3 Total library space per student No of loans per one registered user No of loans per staff member

41. Analyst • Access to all the questionnaires • No modification rights Analysis Questionnaires

42. Selection of the library categories and years Type: academic Other types Source of funding Analyse the years Sort by (years, types)

43. 180 categories for the analysis found, please select Staff Collection

44. Indicators calculated automaticaly Total library space per one user No of loans per one registered user

45. Research libraries performance in 2003 Total library space per user No of loans per one registered user

46. Research libraries performance in 2004 No of loans per one registered user No of loans per staff member

48. Administartor’s Module Libraries Users Questionnaires Forms Outstanding questionnaires Send e-mail Change password Database statistics logout

49. Registration form Send via snail mail to….

50. Add / modify library Sortowanie Sort by Source of funding Type Name