Measuring Innovation in Libraries and Information Services Sebastian Mundt

1. Measuring Innovation in Libraries and Information Services Sebastian Mundt Stuttgart Media University 7th Northumbria International Conference on Performance Measurement in Libraries and Information Services Stellenbosch – Aug. 15, 2007

2. Overview • Drivers of innovation in libraries • Innovation as an object of measurement • Potentials and shortfalls of existing frameworks • Need for additions and improvement • Proposed framework for measuring innovation

3. Drivers of Innovation in Libraries • Increasingly high innovation rate in the media field • Pressure on libraries to compete in order to „remain visible“ • Libraries deliver value added on media • Dependency to provide technological and organizational solutions • Libraries invest in evolving media and technologies • Are these „capital expenditures“ justified and worthwile?

4. Definition of „Innovation“ • Defined in different contexts • often used interchangeably with creativity, flexibility, future potential, invention or improvement “Innovation is the embodiment, combination, or synthesis of knowledge in original, relevant, valued new products, processes, or services” (Luecke/Katz 2003) new or changed substantially, i.e.must add significant value in functional characteristics, ease of use, interaction, sourcing of supplies, or any other dimension successfulin the market

5. Imitation Introduction of a product or process • as a reaction on a specific innovation of another institution • which is aiming at similar markets or addressing similar audiences • where critical technologies or processes are copied from the original innovation generated by the other institution.

6. Examples: Service/process innovations Online interlibrary lending • customers can order 24/7 • quicker delivery • automated status reports • easier handling of orders Implementation of RFID • quicker check-out / shorter waiting times • less check-out errors • better protection of materials against theft • longer life period than conventional barcodes

7. Example: Service innovation Search interface for e-resources • higher comfort in searching different resources • likely more useful search results Example for an imitation: Catalogue enrichment • supports users‘ choice of media • first introduced by Online Bookstores

8. Task StrategicAims Objectives Activities Measures in context of strategic management Select, organize and make literature and information visible for users Make all relevant resources available electronically Provide users with electronic versions of current serials Replace print subscriptions by electronic licenses as far as possible Percentage of total serials expenditure spent on electronic resources

9. Existing frameworks (1) Performance indicators • set of standardized statistics and performance measures • „Objectives of library statistics can be defined as … to monitor trends over time and the results of innovation“ [ISO 2789:2003] • mostly deal with established products and services • outcome and impact indicators not widely tested yet • developments are usually compared over time

10. Resources Potentials Usage Efficiency Existing frameworks (2) Balanced Scorecard • BSC indicators usually focus the institutional perspective • „Potentials and development“ • Percentage of library means received by special grant or income generated • Percentage of library staff providing electronic services • measure development potential, but not actual innovative performance

11. Measuring innovation as a process Phase 1:Creativity Phase 2: Development Phase 3:Marketing Potentials Finances Personnel Efficiency Time budget Financial budget Research output Articles/Papers Performance Functionalities Cost reduction Market relevance Population reached user satisfaction higher output = more value to users more input =more innovative output (likely) higher efficiency = more success(likely) Control resource allocation Provide feedbackto staff Document valueto funding inst.

12. Measurement aims Institution-based • to measure how innovative a library is Project-based • to measure the success of an innovation in terms of input, processes and results

13. Indicators to describe Potentials (1) Number of R&D staff members per year (per innovation project) • Library staff allocated part or full time to one or more innovation projects • in management, researching, creative, developing or administrative functions • Human resources external to the library should be excluded (but included in R&D expenditure). • counted in Full Time Equivalents (FTE) • indicates the workload put into a specific project

14. Indicators to describe Potentials (2) Project expenditure as a percentage of library means • broken down per year per innovation project • including relevant capital expenditure, R&D staff costs (FTE, actual or normalized) including travel costs, IT and consulting expenditure • indicates the financial resources put into a specific innovation project • cf. definition of capital expenditure in ISO 2789:2003

15. Indicators to measure development (processes) Adherence to cost projection Adherence to time schedule • relates actual with pre-planned resource input • could be broken down to single phases of an innovation project if intermediate assessment is required • indicates how likely it is that the project will be finished within the pre-planned resource allocation • however: at least in publicly funded projects, project funding and time schedules are often planned precisely in advance.

16. Measuring success: Difficulties Multi-causality • result(value for an indicator) depends on various management decisions and/or on different external factors Attribution • one innovation can have several impacts • an innovation can trigger mutual dependencies Time lag • to many users, mid or long-term impact becomes visible only after an introductory period of unknown duration (e.g. e-books lifecycle)

17. Indicators of success: Performance Specific service performance, e.g. • Portal: Percentage of titles directly searchable in the portal • RFID: check-outs per member of lending staff, check-out error rate, percentage of lost or misplaced items on shelf • Online ILL: Speed of interlibrary lending (ISO 11620) • mostly specific indicators (needing careful preparation) • compared over time at a pre-defined interval • indicates service and/or process improvement Cost reduction • no standardized measures available • indicates financial resources saved by the innovation

18. Indicators of success: Relevance Percentage of population reached with the service Customer satisfaction regarding specific (survey) items Number of libraries imitating the service • at pre-defined interval • indicate the market relevance of an innovation • usually not relevant for process innovations

19. Summary: Simple working steps • Before starting the project, describe the core functionalities of the innovation. • Define a set of indicators suitable to evaluate the functionalities. • Set their target values. • Agree upon an evaluation deadline. • During and/or after the project, measure. • Compare the results with your target values. (This procedure also works for the assessment of “normal” improvements.)

20. Measurement aims Institution-based • to measure how innovative a library is Project-based • to measure the success of an innovation

21. Assessing the innovative institution (1) Library staff with research and development tasks as a percentage of total library staff • staff allocated to innovation projects of the library or that the library contributes to (e.g. in field tests) • counted in „Full Time Equivalents“ (FTE) • per year • alternatively count in terms of standardized staff costs • similar: Percentage of library staff providing electronic services (ISO 11620)

22. Assessing the innovative institution (2) Expenditure on R&D as a percentage of library means • all means allocated to innovation projects of the library or that the library contributes to • including relevant capital expenditure, R&D staff costs (FTE, actual or normalized) including travel costs, IT and consulting expenditure • broken down per year • similar: Percentage of library means received by special grant or income generated (ISO 11620)

23. Assessing the innovative institution (3) Refereed articles and conference papers per member of R&D staff • per year for all innovation projects • Book publications excluded (results will usually have been published in articles before) • could also be counted in relation to one innovation project

24. Summary and outlook • Different perspective to established performance measurement • continuity less relevant than short handed success • specific measures apply • Can all libraries afford to be „innovative“? • Large libraries are more likely „carriers of innovation“. • Impact / outcome? • Do „innovative“ libraries have a greater impact on their community than others? • What do “innovative” libraries change?

25. Sebastian Mundt Stuttgart Media UniversityFaculty for Information and CommunicationWolframstraße 32D-70191 Stuttgart Tel. +49 (0)711 25706-263Fax +49 (0)711 25706-300 mundt@hdm-stuttgart.de www.hdm-stuttgart.de/iuk